Open
Conversation
- Seed reusable perf fixtures for server scenarios - Add perf provider adapter and web perf tests - Capture artifacts for virtualization and websocket runs
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
github-actions
bot
added
size:XXL
- Refresh virtualization and websocket perf snapshots - Keep perf baselines in sync with latest run
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
Contributor
ApprovabilityVerdict: Needs human review Diff is too large for automated approval analysis. A human reviewer should evaluate this PR. You can customize Macroscope's approvability policy. Learn more. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for all 3 issues found in the latest run.
- ✅ Fixed: Reset creates duplicate animation frame loops corrupting metrics
- Stored the rAF handle and added cancelAnimationFrame before starting a new loop in reset() to prevent duplicate concurrent animation frame chains.
- ✅ Fixed: Generated test artifacts accidentally committed to repository
- Removed the committed artifact JSON files via git rm --cached and added artifacts/ to .gitignore to prevent future accidental commits.
- ✅ Fixed: Percentile returns null for zero target value
- Added Math.max(0, ...) to clamp the computed index so that target=0 returns sorted[0] (the minimum) instead of sorted[-1] (undefined).
Or push these changes by commenting:
@cursor push 0e9b05e984
Preview (0e9b05e984)
diff --git a/.gitignore b/.gitignore
--- a/.gitignore
+++ b/.gitignore
@@ -21,3 +21,4 @@
.vitest-*
__screenshots__/
.tanstack
+artifacts/
diff --git a/artifacts/perf/virtualization-large_threads-1775077017735/virtualization-large_threads.json b/artifacts/perf/virtualization-large_threads-1775077017735/virtualization-large_threads.json
deleted file mode 100644
--- a/artifacts/perf/virtualization-large_threads-1775077017735/virtualization-large_threads.json
+++ /dev/null
@@ -1,141 +1,0 @@
-{
- "suite": "virtualization",
- "scenarioId": "large_threads",
- "startedAt": "2026-04-01T20:57:00.745Z",
- "completedAt": "2026-04-01T20:57:01.524Z",
- "thresholds": {
- "maxMountedTimelineRows": 140,
- "threadSwitchP50Ms": 250,
- "threadSwitchP95Ms": 500,
- "maxLongTaskMs": 120,
- "maxRafGapMs": 120,
- "burstCompletionMs": 5000,
- "longTasksOver50MsMax": 2
- },
- "summary": {
- "maxMountedTimelineRows": 23,
- "threadSwitchP50Ms": 49.5,
- "threadSwitchP95Ms": 117,
- "maxLongTaskMs": 0,
- "longTasksOver50Ms": 0,
- "maxRafGapMs": 25,
- "burstCompletionMs": null
- },
- "browserMetrics": {
- "actions": [
- {
- "name": "thread-switch-warmup-a",
- "durationMs": 117,
- "startedAtMs": 591,
- "endedAtMs": 708
- },
- {
- "name": "thread-switch-1",
- "durationMs": 46.5,
- "startedAtMs": 709.7999999523163,
- "endedAtMs": 756.2999999523163
- },
- {
- "name": "thread-switch-2",
- "durationMs": 49.5,
- "startedAtMs": 787.6000000238419,
- "endedAtMs": 837.1000000238419
- },
- {
- "name": "thread-switch-3",
- "durationMs": 70.69999992847443,
- "startedAtMs": 840.7000000476837,
- "endedAtMs": 911.3999999761581
- },
- {
- "name": "thread-switch-4",
- "durationMs": 45.89999997615814,
- "startedAtMs": 916,
- "endedAtMs": 961.8999999761581
- },
- {
- "name": "thread-switch-5",
- "durationMs": 46.699999928474426,
- "startedAtMs": 964.6000000238419,
- "endedAtMs": 1011.2999999523163
- },
- {
- "name": "thread-switch-6",
- "durationMs": 49.89999997615814,
- "startedAtMs": 1047.8999999761581,
- "endedAtMs": 1097.7999999523163
- }
- ],
- "longTasks": [],
- "rafGapsMs": [
- 0, 8, 0, 17, 0, 7.300000000000068, 0, 8.799999999999955, 0, 8.200000000000045, 0,
- 9.099999999999909, 0, 7.7000000000000455, 0, 8.399999999999977, 0, 8.800000000000068, 0, 16,
- 0, 8.399999999999977, 0, 9, 0, 8.299999999999955, 0, 7.600000000000023, 0, 17.100000000000023,
- 0, 8.600000000000023, 0, 8.299999999999955, 0, 16.200000000000045, 0, 7.899999999999977, 0,
- 9.299999999999955, 0, 7.399999999999977, 0, 9.200000000000045, 0, 16.700000000000045, 0, 8, 0,
- 8.299999999999955, 0, 7.899999999999977, 0, 8.700000000000045, 0, 17.100000000000023, 0,
- 8.199999999999932, 0, 16, 0, 16.600000000000023, 0, 8.399999999999977, 0, 8.899999999999977,
- 0, 7.800000000000068, 0, 17.399999999999977, 0, 8.299999999999955, 0, 8.300000000000068, 0,
- 7.699999999999932, 0, 16.700000000000045, 0, 8.100000000000023, 0, 17.399999999999977, 0,
- 7.7999999999999545, 0, 8.600000000000136, 0, 8.199999999999818, 0, 8.900000000000091, 0,
- 8.099999999999909, 0, 16.200000000000045, 0, 9.100000000000136, 0, 7.399999999999864, 0, 8.5,
- 0, 9, 0, 25, 0, 7.600000000000136, 0, 9.199999999999818, 0
- ],
- "mountedRowSamples": [
- {
- "label": "heavy-a-open",
- "count": 17,
- "capturedAtMs": 708.5
- },
- {
- "label": "thread-switch-1-rows",
- "count": 17,
- "capturedAtMs": 782
- },
- {
- "label": "thread-switch-2-rows",
- "count": 17,
- "capturedAtMs": 837.7000000476837
- },
- {
- "label": "thread-switch-3-rows",
- "count": 17,
- "capturedAtMs": 914.2000000476837
- },
- {
- "label": "thread-switch-4-rows",
- "count": 17,
- "capturedAtMs": 962.5
- },
- {
- "label": "thread-switch-5-rows",
- "count": 17,
- "capturedAtMs": 1034.1000000238419
- },
- {
- "label": "thread-switch-6-rows",
- "count": 17,
- "capturedAtMs": 1098.3999999761581
- },
- {
- "label": "scroll-start",
- "count": 17,
- "capturedAtMs": 1113.3999999761581
- },
- {
- "label": "scroll-top",
- "count": 23,
- "capturedAtMs": 1148.3999999761581
- },
- {
- "label": "scroll-bottom",
- "count": 17,
- "capturedAtMs": 1164.2000000476837
- }
- ]
- },
- "serverMetrics": null,
- "metadata": {
- "heavyThreadMessageCount": 2000
- }
-}
\ No newline at end of file
diff --git a/artifacts/perf/websocket-application-burst_base-1775077022750/websocket-application-dense_assistant_stream.json b/artifacts/perf/websocket-application-burst_base-1775077022750/websocket-application-dense_assistant_stream.json
deleted file mode 100644
--- a/artifacts/perf/websocket-application-burst_base-1775077022750/websocket-application-dense_assistant_stream.json
+++ /dev/null
@@ -1,139 +1,0 @@
-{
- "suite": "websocket-application",
- "scenarioId": "dense_assistant_stream",
- "startedAt": "2026-04-01T20:57:03.910Z",
- "completedAt": "2026-04-01T20:57:07.151Z",
- "thresholds": {
- "maxMountedTimelineRows": 140,
- "threadSwitchP50Ms": 250,
- "threadSwitchP95Ms": 500,
- "maxLongTaskMs": 120,
- "maxRafGapMs": 120,
- "burstCompletionMs": 5000,
- "longTasksOver50MsMax": 2
- },
- "summary": {
- "maxMountedTimelineRows": 0,
- "threadSwitchP50Ms": 51.199999928474426,
- "threadSwitchP95Ms": 52.39999997615814,
- "maxLongTaskMs": 0,
- "longTasksOver50Ms": 0,
- "maxRafGapMs": 16.200000000000045,
- "burstCompletionMs": 3135
- },
- "browserMetrics": {
- "actions": [
- {
- "name": "thread-switch-burst-nav",
- "durationMs": 51.199999928474426,
- "startedAtMs": 1348.3999999761581,
- "endedAtMs": 1399.5999999046326
- },
- {
- "name": "thread-switch-burst-return",
- "durationMs": 52.39999997615814,
- "startedAtMs": 1401.1999999284744,
- "endedAtMs": 1453.5999999046326
- },
- {
- "name": "burst-completion",
- "durationMs": 3135,
- "startedAtMs": 332.2999999523163,
- "endedAtMs": 3467.2999999523163
- }
- ],
- "longTasks": [],
- "rafGapsMs": [
- 0, 7.899999999999977, 0, 9, 0, 7.300000000000011, 0, 8.899999999999977, 0, 7.800000000000011,
- 0, 9.300000000000011, 0, 8.100000000000023, 0, 7.699999999999989, 0, 9.199999999999989, 0,
- 7.800000000000011, 0, 7.899999999999977, 0, 9.300000000000011, 0, 7.300000000000011, 0,
- 9.099999999999966, 0, 7.7000000000000455, 0, 8.199999999999989, 0, 8.399999999999977, 0,
- 8.300000000000011, 0, 8.399999999999977, 0, 8.5, 0, 9.100000000000023, 0, 7.7000000000000455,
- 0, 8.199999999999932, 0, 8.600000000000023, 0, 8.799999999999955, 0, 7.7000000000000455, 0,
- 8.200000000000045, 0, 8.399999999999977, 0, 9.100000000000023, 0, 7.5, 0, 8.199999999999932,
- 0, 9.300000000000068, 0, 7.399999999999977, 0, 8.600000000000023, 0, 8.399999999999977, 0,
- 8.899999999999977, 0, 8.299999999999955, 0, 7.800000000000068, 0, 8, 0, 8.399999999999977, 0,
- 8.799999999999955, 0, 7.7000000000000455, 0, 9.299999999999955, 0, 7.400000000000091, 0,
- 9.299999999999955, 0, 8.299999999999955, 0, 7.7000000000000455, 0, 8.100000000000023, 0,
- 8.899999999999977, 0, 8.700000000000045, 0, 7.2999999999999545, 0, 9.399999999999977, 0,
- 8.200000000000045, 0, 8.100000000000023, 0, 7.699999999999932, 0, 9.300000000000068, 0,
- 8.199999999999932, 0, 8.300000000000068, 0, 7.699999999999932, 0, 8.700000000000045, 0,
- 8.699999999999932, 0, 7.600000000000023, 0, 8.100000000000023, 0, 8.399999999999977, 0,
- 8.700000000000045, 0, 8, 0, 8.699999999999932, 0, 8.200000000000045, 0, 8.799999999999955, 0,
- 8.5, 0, 7.800000000000068, 0, 8.5, 0, 8.600000000000023, 0, 8.399999999999977, 0,
- 7.899999999999977, 0, 8.100000000000023, 0, 8.100000000000023, 0, 8.399999999999977, 0,
- 9.299999999999955, 0, 8.100000000000023, 0, 7.5, 0, 9.200000000000045, 0, 8.5, 0,
- 7.899999999999864, 0, 8.800000000000182, 0, 8.199999999999818, 0, 7.800000000000182, 0, 8.5,
- 0, 8.199999999999818, 0, 8.900000000000091, 0, 8.299999999999955, 0, 7.900000000000091, 0,
- 7.899999999999864, 0, 9.300000000000182, 0, 7.599999999999909, 0, 8.400000000000091, 0, 8, 0,
- 8.399999999999864, 0, 9.200000000000045, 0, 7.400000000000091, 0, 9.299999999999955, 0,
- 8.200000000000045, 0, 7.5, 0, 9.299999999999955, 0, 8.099999999999909, 0, 7.600000000000136,
- 0, 9.299999999999955, 0, 8.200000000000045, 0, 8.399999999999864, 0, 7.5, 0,
- 8.400000000000091, 0, 8.599999999999909, 0, 7.7999999999999545, 0, 8.5, 0, 9.300000000000182,
- 0, 7.2999999999999545, 0, 9.399999999999864, 0, 7.5, 0, 8.100000000000136, 0, 8.5, 0,
- 8.099999999999909, 0, 9.100000000000136, 0, 8.599999999999909, 0, 7.5, 0, 9.099999999999909,
- 0, 7.600000000000136, 0, 8.399999999999864, 0, 8.100000000000136, 0, 8.399999999999864, 0,
- 8.200000000000045, 0, 9.299999999999955, 0, 8, 0, 16.200000000000045, 0, 9.200000000000045, 0,
- 8.299999999999955, 0, 8.299999999999955, 0, 8.400000000000091, 0, 7.900000000000091, 0,
- 8.599999999999909, 0, 8.5, 0, 8.099999999999909, 0, 7.7000000000000455, 0, 8.700000000000045,
- 0, 8.5, 0, 8.5, 0, 7.599999999999909, 0, 8.900000000000091, 0, 8.599999999999909, 0,
- 8.400000000000091, 0, 8.299999999999955, 0, 7.7999999999999545, 0, 8.600000000000136, 0,
- 8.200000000000045, 0, 8.799999999999955, 0, 7.599999999999909, 0, 8.100000000000136, 0,
- 8.599999999999909, 0, 8.299999999999955, 0, 8.100000000000136, 0, 8.599999999999909, 0,
- 9.099999999999909, 0, 8.200000000000045, 0, 7.400000000000091, 0, 8.899999999999864, 0, 8.5,
- 0, 8.5, 0, 7.900000000000091, 0, 8.799999999999955, 0, 7.400000000000091, 0,
- 9.299999999999955, 0, 7.400000000000091, 0, 9.299999999999955, 0, 8.299999999999955, 0,
- 8.299999999999955, 0, 7.400000000000091, 0, 8.899999999999864, 0, 8.800000000000182, 0,
- 7.599999999999909, 0, 8.099999999999909, 0, 8.200000000000045, 0, 9.400000000000091, 0,
- 7.7000000000000455, 0, 8.899999999999864, 0, 7.400000000000091, 0, 8.399999999999864, 0,
- 8.900000000000091, 0, 8, 0, 8.5, 0, 8.799999999999955, 0, 7.7000000000000455, 0,
- 8.200000000000045, 0, 8.700000000000045, 0, 7.899999999999864, 0, 8.799999999999955, 0,
- 8.400000000000091, 0, 8.099999999999909, 0, 8.200000000000045, 0, 8.200000000000045, 0,
- 8.200000000000045, 0, 8.799999999999955, 0, 7.7999999999999545, 0, 8.600000000000136, 0,
- 8.199999999999818, 0, 9.200000000000045, 0, 7.500000000000227, 0, 8.199999999999818, 0,
- 9.099999999999909, 0, 7.700000000000273, 0, 8.899999999999636, 0, 7.600000000000364, 0,
- 8.399999999999636, 0, 9.300000000000182, 0, 7.400000000000091, 0, 8.299999999999727, 0,
- 9.300000000000182, 0, 8.300000000000182, 0, 7.799999999999727, 0, 8.900000000000091, 0,
- 7.699999999999818, 0, 8.800000000000182, 0, 8.300000000000182, 0, 7.599999999999909, 0,
- 8.799999999999727, 0, 7.900000000000091, 0, 9.099999999999909, 0, 8.200000000000273, 0, 8, 0,
- 8.799999999999727, 0, 7.600000000000364, 0, 9.299999999999727, 0, 8.200000000000273, 0,
- 7.699999999999818, 0, 8, 0, 8.599999999999909, 0, 9.099999999999909, 0, 7.300000000000182, 0,
- 8.900000000000091, 0, 8.799999999999727, 0, 8.100000000000364, 0, 7.899999999999636, 0, 8, 0,
- 8.400000000000091, 0, 9.099999999999909, 0, 7.5, 0, 8.800000000000182, 0, 7.900000000000091,
- 0, 9.299999999999727, 0, 7.5, 0, 9.100000000000364, 0, 8.099999999999909, 0,
- 7.799999999999727, 0, 8.700000000000273, 0, 7.799999999999727, 0, 9.300000000000182, 0,
- 8.300000000000182, 0, 8, 0, 8.699999999999818, 0, 7.599999999999909, 0, 9.099999999999909, 0,
- 8.200000000000273, 0, 7.599999999999909, 0, 9.199999999999818, 0, 8.200000000000273, 0,
- 8.400000000000091, 0, 8.299999999999727, 0, 8.200000000000273, 0, 8.5, 0, 8.299999999999727,
- 0, 8.400000000000091, 0, 8.300000000000182, 0, 7.5, 0, 8.5, 0, 8.099999999999909, 0,
- 9.299999999999727, 0, 7.400000000000091, 0, 9, 0, 8, 0, 8.900000000000091, 0,
- 8.300000000000182, 0, 8.299999999999727, 0, 7.5, 0, 8.599999999999909, 0, 8, 0,
- 9.300000000000182, 0, 7.400000000000091, 0, 9.299999999999727, 0, 8.300000000000182, 0,
- 7.599999999999909, 0, 8.800000000000182, 0, 8.099999999999909, 0, 8, 0, 8.800000000000182, 0,
- 8.5, 0, 8.5, 0, 8.099999999999909, 0, 8.5, 0, 8, 0, 8.199999999999818, 0, 7.900000000000091,
- 0, 8.599999999999909, 0, 8.599999999999909, 0, 8, 0, 9.100000000000364, 0, 8.199999999999818,
- 0, 8.5, 0, 8, 0, 7.800000000000182, 0, 8.899999999999636, 0, 7.800000000000182, 0,
- 9.099999999999909, 0, 8.300000000000182, 0, 7.799999999999727, 0, 9, 0, 8.300000000000182, 0,
- 7.5, 0, 9.199999999999818, 0, 7.400000000000091, 0, 9.300000000000182, 0, 8, 0,
- 7.799999999999727, 0, 9.200000000000273, 0, 8, 0, 7.599999999999909, 0, 8.400000000000091, 0,
- 9.299999999999727, 0, 7.300000000000182, 0, 8.5, 0, 9.199999999999818, 0, 8.200000000000273,
- 0, 8.400000000000091, 0, 7.5, 0, 9.199999999999818, 0, 8.300000000000182, 0,
- 7.399999999999636, 0, 8.400000000000091, 0, 9, 0, 8.599999999999909, 0, 8.200000000000273, 0,
- 7.5, 0, 9.299999999999727, 0, 8.200000000000273, 0, 8.400000000000091, 0, 7.399999999999636,
- 0, 8.300000000000182, 0, 9.400000000000091, 0, 8, 0, 8.5, 0, 8.099999999999909, 0, 8, 0, 8, 0,
- 8.900000000000091, 0, 7.900000000000091, 0, 8.299999999999727, 0, 8.300000000000182, 0, 8.5,
- 0, 8.699999999999818, 0, 8.099999999999909, 0, 9, 0, 8.300000000000182, 0, 7.400000000000091,
- 0, 9.099999999999909, 0, 7.800000000000182, 0, 9.099999999999909, 0, 8.199999999999818, 0,
- 8.400000000000091, 0, 8.300000000000182, 0, 7.899999999999636, 0, 8.700000000000273, 0, 8.5,
- 0, 7.400000000000091, 0, 8.299999999999727, 0, 8.300000000000182, 0, 9.299999999999727, 0,
- 8.300000000000182, 0, 8.199999999999818, 0
- ],
- "mountedRowSamples": []
- },
- "serverMetrics": null,
- "metadata": {
- "burstSeedThreadId": "perf-thread-burst",
- "navigationThreadId": "perf-thread-light-01",
- "sentinelText": "PERF_STREAM_SENTINEL:dense_assistant_stream:completed"
- }
-}
\ No newline at end of file
diff --git a/test/perf/support/artifact.ts b/test/perf/support/artifact.ts
--- a/test/perf/support/artifact.ts
+++ b/test/perf/support/artifact.ts
@@ -68,7 +68,10 @@
}
const sorted = values.toSorted((left, right) => left - right);
const clampedTarget = Math.min(Math.max(target, 0), 1);
- const index = Math.min(sorted.length - 1, Math.ceil(sorted.length * clampedTarget) - 1);
+ const index = Math.min(
+ sorted.length - 1,
+ Math.max(0, Math.ceil(sorted.length * clampedTarget) - 1),
+ );
return sorted[index] ?? null;
}
diff --git a/test/perf/support/browserMetrics.ts b/test/perf/support/browserMetrics.ts
--- a/test/perf/support/browserMetrics.ts
+++ b/test/perf/support/browserMetrics.ts
@@ -30,15 +30,16 @@
const rafGapsMs: number[] = [];
const mountedRowSamples: Array<BrowserPerfMetrics["mountedRowSamples"][number]> = [];
let previousAnimationFrameTs = 0;
+ let rafHandle = 0;
const animationFrameLoop = (timestampMs: number) => {
if (previousAnimationFrameTs > 0) {
rafGapsMs.push(timestampMs - previousAnimationFrameTs);
}
previousAnimationFrameTs = timestampMs;
- window.requestAnimationFrame(animationFrameLoop);
+ rafHandle = window.requestAnimationFrame(animationFrameLoop);
};
- window.requestAnimationFrame(animationFrameLoop);
+ rafHandle = window.requestAnimationFrame(animationFrameLoop);
if (typeof PerformanceObserver !== "undefined") {
try {
@@ -103,7 +104,8 @@
rafGapsMs.length = 0;
mountedRowSamples.length = 0;
previousAnimationFrameTs = 0;
- window.requestAnimationFrame(animationFrameLoop);
+ window.cancelAnimationFrame(rafHandle);
+ rafHandle = window.requestAnimationFrame(animationFrameLoop);
},
};
}You can send follow-ups to this agent here.
artifacts/perf/virtualization-large_threads-1775077017735/virtualization-large_threads.json
Outdated
Show resolved
Hide resolved
- Expand perf fixtures with multi-thread live stream events and namespaced IDs - Harden stream getters and update web perf tests for the heavier workload
juliusmarminge
force-pushed
the
t3code/performance-regression-tests
branch
from
b9a8795 to
18f19a6
Compare
macroscopeapp
bot
dismissed
their stale review
Dismissing prior approval to re-evaluate 18f19a6
- Keep `bun run test` focused on non-perf suites
- Add README entry for perf benchmarks - Document scenarios, commands, artifacts, and env vars
- Persist and remove the seeded run parent directory after perf runs - Add regression coverage for seed cleanup, percentile clamping, and RAF reset
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Perf provider always enabled even without provider scenario
- Moved PERF_PROVIDER_ENV inside the providerScenarioId conditional so it is only set when a provider scenario is specified, matching the behavior in open-perf-app.ts.
Or push these changes by commenting:
@cursor push e26c2abd97
Preview (e26c2abd97)
diff --git a/apps/web/test/perf/appHarness.ts b/apps/web/test/perf/appHarness.ts
--- a/apps/web/test/perf/appHarness.ts
+++ b/apps/web/test/perf/appHarness.ts
@@ -268,8 +268,12 @@
const env = {
...process.env,
T3CODE_AUTO_BOOTSTRAP_PROJECT_FROM_CWD: "false",
- [PERF_PROVIDER_ENV]: "1",
- ...(options.providerScenarioId ? { [PERF_SCENARIO_ENV]: options.providerScenarioId } : {}),
+ ...(options.providerScenarioId
+ ? {
+ [PERF_PROVIDER_ENV]: "1",
+ [PERF_SCENARIO_ENV]: options.providerScenarioId,
+ }
+ : {}),
};
let stdoutBuffer = "";You can send follow-ups to this agent here.
- Seed large_threads across 5 projects with bounded heavy turns - Add shared perf server env setup and update harness assertions - Co-authored-by: codex <codex@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Failed template creation cached permanently as rejected promise
- Added a .catch handler on the createTemplateDir promise that evicts the entry from templateDirPromises on rejection, allowing subsequent calls to retry.
- ✅ Fixed: Duplicated utility functions across harness and CLI script
- Extracted pickFreePort, waitForServerReady, stopChildProcess, cleanupPerfRunDir, verifyBuiltArtifacts, and parsePerfSeededState into a shared test/perf/support/perfProcess.ts module and updated both consumers to import from it.
Or push these changes by commenting:
@cursor push 89497fed15
Preview (89497fed15)
diff --git a/apps/server/integration/perf/seedPerfState.ts b/apps/server/integration/perf/seedPerfState.ts
--- a/apps/server/integration/perf/seedPerfState.ts
+++ b/apps/server/integration/perf/seedPerfState.ts
@@ -546,7 +546,10 @@
if (existing) {
return existing;
}
- const created = createTemplateDir(scenarioId);
+ const created = createTemplateDir(scenarioId).catch((error: unknown) => {
+ templateDirPromises.delete(scenarioId);
+ throw error;
+ });
templateDirPromises.set(scenarioId, created);
return created;
}
diff --git a/apps/web/test/perf/appHarness.ts b/apps/web/test/perf/appHarness.ts
--- a/apps/web/test/perf/appHarness.ts
+++ b/apps/web/test/perf/appHarness.ts
@@ -1,6 +1,5 @@
-import { spawn, type ChildProcess } from "node:child_process";
-import { access, mkdir, rm, writeFile } from "node:fs/promises";
-import { createServer } from "node:net";
+import { spawn } from "node:child_process";
+import { mkdir, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { once } from "node:events";
@@ -18,6 +17,14 @@
installBrowserPerfCollector,
PERF_BROWSER_GLOBAL,
} from "../../../../test/perf/support/browserMetrics";
+import {
+ pickFreePort,
+ waitForServerReady,
+ stopChildProcess,
+ cleanupPerfRunDir,
+ verifyBuiltArtifacts,
+ parsePerfSeededState,
+} from "../../../../test/perf/support/perfProcess";
import type { PerfThresholdProfile } from "../../../../test/perf/support/thresholds";
import type {
PerfProviderScenarioId,
@@ -35,8 +42,6 @@
const serverClientIndexPath = resolve(repoRoot, "apps/server/dist/client/index.html");
const PERF_ARTIFACT_DIR_ENV = "T3CODE_PERF_ARTIFACT_DIR";
const PERF_HEADFUL_ENV = "T3CODE_PERF_HEADFUL";
-const PERF_SEED_JSON_START = "__T3_PERF_SEED_JSON_START__";
-const PERF_SEED_JSON_END = "__T3_PERF_SEED_JSON_END__";
interface PerfSeedThreadSummary {
readonly id: string;
@@ -104,84 +109,6 @@
}>;
}
-async function pickFreePort(): Promise<number> {
- return await new Promise<number>((resolvePort, reject) => {
- const server = createServer();
- server.on("error", reject);
- server.listen(0, "127.0.0.1", () => {
- const address = server.address();
- if (!address || typeof address === "string") {
- reject(new Error("Unable to resolve a free localhost port."));
- return;
- }
- const { port } = address;
- server.close((closeError) => {
- if (closeError) {
- reject(closeError);
- return;
- }
- resolvePort(port);
- });
- });
- });
-}
-
-async function waitForServerReady(url: string, process: ChildProcess): Promise<void> {
- const startedAt = Date.now();
- const timeoutMs = 45_000;
- const requestTimeoutMs = 1_000;
-
- while (Date.now() - startedAt < timeoutMs) {
- if (process.exitCode !== null) {
- throw new Error(`Perf server exited early with code ${process.exitCode}.`);
- }
- try {
- const response = await fetch(url, {
- redirect: "manual",
- signal: AbortSignal.timeout(requestTimeoutMs),
- });
- if (response.ok) {
- return;
- }
- } catch {
- // Ignore connection races while the server is still starting.
- }
- await new Promise((resolveDelay) => setTimeout(resolveDelay, 200));
- }
-
- throw new Error(`Timed out waiting for perf server readiness at ${url}.`);
-}
-
-async function verifyBuiltArtifacts(): Promise<void> {
- await Promise.all([access(serverBinPath), access(serverClientIndexPath)]).catch(() => {
- throw new Error(
- `Built perf artifacts are missing. Expected ${serverBinPath} and ${serverClientIndexPath}. Run bun run test:perf:web or build the app first.`,
- );
- });
-}
-
-async function stopChildProcess(process: ChildProcess): Promise<void> {
- if (process.exitCode !== null) {
- return;
- }
-
- process.kill("SIGTERM");
- const exited = await new Promise<boolean>((resolveExited) => {
- const timer = setTimeout(() => resolveExited(false), 5_000);
- process.once("exit", () => {
- clearTimeout(timer);
- resolveExited(true);
- });
- });
-
- if (!exited && process.exitCode === null) {
- process.kill("SIGKILL");
- await new Promise<void>((resolveExited) => {
- process.once("exit", () => resolveExited());
- });
- }
-}
-
async function ensureArtifactDir(suite: string, scenarioId: string): Promise<string> {
const baseArtifactDir = resolve(
process.env[PERF_ARTIFACT_DIR_ENV] ?? join(repoRoot, "artifacts/perf"),
@@ -192,10 +119,6 @@
return artifactDir;
}
-async function cleanupPerfRunDir(runParentDir: string): Promise<void> {
- await rm(runParentDir, { recursive: true, force: true });
-}
-
async function writeServerLogs(
artifactDir: string,
stdout: string,
@@ -231,22 +154,10 @@
);
}
-function parsePerfSeededState(stdout: string): PerfSeededState {
- const startIndex = stdout.lastIndexOf(PERF_SEED_JSON_START);
- const endIndex = stdout.lastIndexOf(PERF_SEED_JSON_END);
-
- if (startIndex !== -1 && endIndex !== -1 && endIndex > startIndex) {
- const payload = stdout.slice(startIndex + PERF_SEED_JSON_START.length, endIndex).trim();
- return JSON.parse(payload) as PerfSeededState;
- }
-
- return JSON.parse(stdout) as PerfSeededState;
-}
-
export async function startPerfAppHarness(
options: StartPerfAppHarnessOptions,
): Promise<PerfAppHarness> {
- await verifyBuiltArtifacts();
+ await verifyBuiltArtifacts([serverBinPath, serverClientIndexPath]);
const seededState = await (async () => {
const seedProcess = spawn(
@@ -270,7 +181,7 @@
if (exitCode !== 0) {
throw new Error(`Perf seed command failed with code ${exitCode ?? "unknown"}.\n${stderr}`);
}
- return parsePerfSeededState(stdout);
+ return parsePerfSeededState<PerfSeededState>(stdout);
})();
const artifactDir = await ensureArtifactDir(options.suite, options.seedScenarioId);
const port = await pickFreePort();
diff --git a/apps/web/tsconfig.json b/apps/web/tsconfig.json
--- a/apps/web/tsconfig.json
+++ b/apps/web/tsconfig.json
@@ -27,6 +27,7 @@
"test",
"../../test/perf/support/artifact.ts",
"../../test/perf/support/browserMetrics.ts",
+ "../../test/perf/support/perfProcess.ts",
"../../test/perf/support/serverSampler.ts",
"../../test/perf/support/thresholds.ts"
]
diff --git a/scripts/open-perf-app.ts b/scripts/open-perf-app.ts
--- a/scripts/open-perf-app.ts
+++ b/scripts/open-perf-app.ts
@@ -1,17 +1,22 @@
-import { spawn, type ChildProcess } from "node:child_process";
+import { spawn } from "node:child_process";
import { once } from "node:events";
-import { access, rm } from "node:fs/promises";
-import { createServer } from "node:net";
import { resolve } from "node:path";
import { fileURLToPath } from "node:url";
+import {
+ pickFreePort,
+ waitForServerReady,
+ stopChildProcess,
+ cleanupPerfRunDir,
+ verifyBuiltArtifacts,
+ parsePerfSeededState,
+} from "../test/perf/support/perfProcess";
+
const repoRoot = fileURLToPath(new URL("../", import.meta.url));
const serverBinPath = resolve(repoRoot, "apps/server/dist/bin.mjs");
const serverClientIndexPath = resolve(repoRoot, "apps/server/dist/client/index.html");
const PERF_PROVIDER_ENV = "T3CODE_PERF_PROVIDER";
const PERF_SCENARIO_ENV = "T3CODE_PERF_SCENARIO";
-const PERF_SEED_JSON_START = "__T3_PERF_SEED_JSON_START__";
-const PERF_SEED_JSON_END = "__T3_PERF_SEED_JSON_END__";
type PerfSeedScenarioId = "large_threads" | "burst_base";
type PerfProviderScenarioId = "dense_assistant_stream";
@@ -151,47 +156,6 @@
};
}
-async function pickFreePort(): Promise<number> {
- return await new Promise<number>((resolvePort, reject) => {
- const server = createServer();
- server.on("error", reject);
- server.listen(0, "127.0.0.1", () => {
- const address = server.address();
- if (!address || typeof address === "string") {
- reject(new Error("Unable to resolve a free localhost port."));
- return;
- }
- server.close((closeError) => {
- if (closeError) {
- reject(closeError);
- return;
- }
- resolvePort(address.port);
- });
- });
- });
-}
-
-async function verifyBuiltArtifacts(): Promise<void> {
- await Promise.all([access(serverBinPath), access(serverClientIndexPath)]).catch(() => {
- throw new Error(
- `Built perf artifacts are missing. Expected ${serverBinPath} and ${serverClientIndexPath}. Run bun run test:perf:web or build the app first.`,
- );
- });
-}
-
-function parsePerfSeededState(stdout: string): PerfSeededState {
- const startIndex = stdout.lastIndexOf(PERF_SEED_JSON_START);
- const endIndex = stdout.lastIndexOf(PERF_SEED_JSON_END);
-
- if (startIndex === -1 || endIndex === -1 || endIndex <= startIndex) {
- throw new Error(`Perf seed command did not emit the expected JSON markers.\n${stdout}`);
- }
-
- const payload = stdout.slice(startIndex + PERF_SEED_JSON_START.length, endIndex).trim();
- return JSON.parse(payload) as PerfSeededState;
-}
-
async function seedPerfState(scenarioId: PerfSeedScenarioId): Promise<PerfSeededState> {
const seedProcess = spawn("bun", ["run", "apps/server/scripts/seedPerfState.ts", scenarioId], {
cwd: repoRoot,
@@ -213,61 +177,9 @@
throw new Error(`Perf seed command failed with code ${exitCode ?? "unknown"}.\n${stderr}`);
}
- return parsePerfSeededState(stdout);
+ return parsePerfSeededState<PerfSeededState>(stdout);
}
-async function waitForServerReady(url: string, process: ChildProcess): Promise<void> {
- const startedAt = Date.now();
- const timeoutMs = 45_000;
- const requestTimeoutMs = 1_000;
-
- while (Date.now() - startedAt < timeoutMs) {
- if (process.exitCode !== null) {
- throw new Error(`Perf server exited early with code ${process.exitCode}.`);
- }
- try {
- const response = await fetch(url, {
- redirect: "manual",
- signal: AbortSignal.timeout(requestTimeoutMs),
- });
- if (response.ok) {
- return;
- }
- } catch {
- // Ignore connection races during startup.
- }
- await new Promise((resolveDelay) => setTimeout(resolveDelay, 200));
- }
-
- throw new Error(`Timed out waiting for perf server readiness at ${url}.`);
-}
-
-async function stopChildProcess(process: ChildProcess): Promise<void> {
- if (process.exitCode !== null) {
- return;
- }
-
- process.kill("SIGTERM");
- const exited = await new Promise<boolean>((resolveExited) => {
- const timer = setTimeout(() => resolveExited(false), 5_000);
- process.once("exit", () => {
- clearTimeout(timer);
- resolveExited(true);
- });
- });
-
- if (!exited && process.exitCode === null) {
- process.kill("SIGKILL");
- await new Promise<void>((resolveExited) => {
- process.once("exit", () => resolveExited());
- });
- }
-}
-
-async function cleanupPerfRunDir(runParentDir: string): Promise<void> {
- await rm(runParentDir, { recursive: true, force: true });
-}
-
function openUrl(url: string): void {
const command: [string, ...string[]] =
process.platform === "darwin"
@@ -320,7 +232,7 @@
async function main(): Promise<void> {
const options = parseArgs(process.argv.slice(2));
- await verifyBuiltArtifacts();
+ await verifyBuiltArtifacts([serverBinPath, serverClientIndexPath]);
const seededState = await seedPerfState(options.scenarioId);
const port = options.port === 0 ? await pickFreePort() : options.port;
diff --git a/scripts/tsconfig.json b/scripts/tsconfig.json
--- a/scripts/tsconfig.json
+++ b/scripts/tsconfig.json
@@ -12,5 +12,5 @@
}
]
},
- "include": ["**/*.ts"]
+ "include": ["**/*.ts", "../test/perf/support/perfProcess.ts"]
}
diff --git a/test/perf/support/perfProcess.ts b/test/perf/support/perfProcess.ts
new file mode 100644
--- /dev/null
+++ b/test/perf/support/perfProcess.ts
@@ -1,0 +1,100 @@
+import { type ChildProcess } from "node:child_process";
+import { access, rm } from "node:fs/promises";
+import { createServer } from "node:net";
+
+const PERF_SEED_JSON_START = "__T3_PERF_SEED_JSON_START__";
+const PERF_SEED_JSON_END = "__T3_PERF_SEED_JSON_END__";
+
+export async function pickFreePort(): Promise<number> {
+ return await new Promise<number>((resolvePort, reject) => {
+ const server = createServer();
+ server.on("error", reject);
+ server.listen(0, "127.0.0.1", () => {
+ const address = server.address();
+ if (!address || typeof address === "string") {
+ reject(new Error("Unable to resolve a free localhost port."));
+ return;
+ }
+ const { port } = address;
+ server.close((closeError) => {
+ if (closeError) {
+ reject(closeError);
+ return;
+ }
+ resolvePort(port);
+ });
+ });
+ });
+}
+
+export async function waitForServerReady(url: string, process: ChildProcess): Promise<void> {
+ const startedAt = Date.now();
+ const timeoutMs = 45_000;
+ const requestTimeoutMs = 1_000;
+
+ while (Date.now() - startedAt < timeoutMs) {
+ if (process.exitCode !== null) {
+ throw new Error(`Perf server exited early with code ${process.exitCode}.`);
+ }
+ try {
+ const response = await fetch(url, {
+ redirect: "manual",
+ signal: AbortSignal.timeout(requestTimeoutMs),
+ });
+ if (response.ok) {
+ return;
+ }
+ } catch {
+ // Ignore connection races while the server is still starting.
+ }
+ await new Promise((resolveDelay) => setTimeout(resolveDelay, 200));
+ }
+
+ throw new Error(`Timed out waiting for perf server readiness at ${url}.`);
+}
+
+export async function stopChildProcess(process: ChildProcess): Promise<void> {
+ if (process.exitCode !== null) {
+ return;
+ }
+
+ process.kill("SIGTERM");
+ const exited = await new Promise<boolean>((resolveExited) => {
+ const timer = setTimeout(() => resolveExited(false), 5_000);
+ process.once("exit", () => {
+ clearTimeout(timer);
+ resolveExited(true);
+ });
+ });
+
+ if (!exited && process.exitCode === null) {
+ process.kill("SIGKILL");
+ await new Promise<void>((resolveExited) => {
+ process.once("exit", () => resolveExited());
+ });
+ }
+}
+
+export async function cleanupPerfRunDir(runParentDir: string): Promise<void> {
+ await rm(runParentDir, { recursive: true, force: true });
+}
+
+export async function verifyBuiltArtifacts(paths: ReadonlyArray<string>): Promise<void> {
+ await Promise.all(paths.map((p) => access(p))).catch(() => {
+ throw new Error(
+ `Built perf artifacts are missing. Expected ${paths.join(" and ")}. Run bun run test:perf:web or build the app first.`,
+ );
+ });
+}
+
+export function parsePerfSeededState<T>(stdout: string): T {
+ const startIndex = stdout.lastIndexOf(PERF_SEED_JSON_START);
+ const endIndex = stdout.lastIndexOf(PERF_SEED_JSON_END);
+
+ if (startIndex !== -1 && endIndex !== -1 && endIndex > startIndex) {
+ const payload = stdout.slice(startIndex + PERF_SEED_JSON_START.length, endIndex).trim();
+ return JSON.parse(payload) as T;
+ }
+
+ return JSON.parse(stdout) as T;
+}You can send follow-ups to this agent here.
| const created = createTemplateDir(scenarioId); | ||
| templateDirPromises.set(scenarioId, created); | ||
| return created; | ||
| } |
Contributor
There was a problem hiding this comment.
Failed template creation cached permanently as rejected promise
Low Severity
getTemplateDir caches the promise returned by createTemplateDir in the module-level templateDirPromises map before it resolves. If createTemplateDir fails (e.g., git not found, disk error), the rejected promise is permanently cached, so every subsequent call for the same scenarioId will immediately return the same rejection instead of retrying. This could cause confusing cascading failures when running multiple perf tests in the same process.
| process.once("exit", () => resolveExited()); | ||
| }); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Duplicated utility functions across harness and CLI script
Low Severity
pickFreePort, waitForServerReady, stopChildProcess, cleanupPerfRunDir, verifyBuiltArtifacts, and parsePerfSeededState are duplicated nearly verbatim between scripts/open-perf-app.ts and apps/web/test/perf/appHarness.ts. These could be extracted to a shared module to avoid divergent fixes over time.
Additional Locations (1)
| ); | ||
| const runtime = ManagedRuntime.make(seedLayer); | ||
|
|
||
| const snapshot = await runtime.runPromise( |
Contributor
There was a problem hiding this comment.
🟡 Medium perf/seedPerfState.ts:503
If runtime.runPromise(...) throws (e.g., from eventStore.append or projectionPipeline.projectEvent), runtime.dispose() on line 540 is never called. This leaks the ManagedRuntime resources including the SQLite connection, and because templateDirPromises caches the promise, the leaked runtime persists for the process lifetime. Consider using Effect.tapError or a try/finally pattern to ensure dispose() runs even on failure.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file apps/server/integration/perf/seedPerfState.ts around line 503:
If `runtime.runPromise(...)` throws (e.g., from `eventStore.append` or `projectionPipeline.projectEvent`), `runtime.dispose()` on line 540 is never called. This leaks the `ManagedRuntime` resources including the SQLite connection, and because `templateDirPromises` caches the promise, the leaked runtime persists for the process lifetime. Consider using `Effect.tapError` or a `try/finally` pattern to ensure `dispose()` runs even on failure.
Evidence trail:
apps/server/integration/perf/seedPerfState.ts lines 473-541 (createTemplateDir function): Line 502 creates runtime via `ManagedRuntime.make(seedLayer)`, line 504-522 calls `runtime.runPromise(...)`, line 540 calls `runtime.dispose()`. No try/finally block exists. Line 46 shows module-level cache `templateDirPromises = new Map<PerfSeedScenarioId, Promise<string>>();`. Lines 543-552 show `getTemplateDir` caches the promise before resolution.
Co-authored-by: codex <codex@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 4 total unresolved issues (including 2 from previous reviews).
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Seed process stdout may be incomplete due to 'exit' event
- Changed
once(seedProcess, "exit")toonce(seedProcess, "close")in bothappHarness.tsandopen-perf-app.tsto ensure piped stdio buffers are fully drained before parsing stdout.
- Changed
- ✅ Fixed: Duplicated
buildPerfServerEnvacross server and web harnesses- Extracted
buildPerfServerEnvand its env-var constants into a new shared module at@t3tools/shared/perf/serverEnvand updated both consumers to import from it.
- Extracted
Or push these changes by commenting:
@cursor push 82b826072f
Preview (82b826072f)
diff --git a/apps/server/integration/perf/serverPerfHarness.ts b/apps/server/integration/perf/serverPerfHarness.ts
--- a/apps/server/integration/perf/serverPerfHarness.ts
+++ b/apps/server/integration/perf/serverPerfHarness.ts
@@ -33,13 +33,11 @@
PerfProviderScenarioId,
PerfSeedScenarioId,
} from "@t3tools/shared/perf/scenarioCatalog";
+import { buildPerfServerEnv } from "@t3tools/shared/perf/serverEnv";
import { seedPerfState, type PerfSeededState } from "./seedPerfState.ts";
const repoRoot = fileURLToPath(new URL("../../../../", import.meta.url));
const PERF_ARTIFACT_DIR_ENV = "T3CODE_PERF_ARTIFACT_DIR";
-const PERF_PROVIDER_ENV = "T3CODE_PERF_PROVIDER";
-const PERF_SCENARIO_ENV = "T3CODE_PERF_SCENARIO";
-const AUTO_BOOTSTRAP_PROJECT_ENV = "T3CODE_AUTO_BOOTSTRAP_PROJECT_FROM_CWD";
const makeWsRpcClient = RpcClient.make(WsRpcGroup);
type WsRpcClient =
@@ -184,28 +182,6 @@
]);
}
-function buildPerfServerEnv(
- baseEnv: NodeJS.ProcessEnv,
- providerScenarioId?: PerfProviderScenarioId,
-): NodeJS.ProcessEnv {
- const env: NodeJS.ProcessEnv = {
- ...baseEnv,
- [AUTO_BOOTSTRAP_PROJECT_ENV]: "false",
- };
-
- if (!providerScenarioId) {
- delete env[PERF_PROVIDER_ENV];
- delete env[PERF_SCENARIO_ENV];
- return env;
- }
-
- return {
- ...env,
- [PERF_PROVIDER_ENV]: "1",
- [PERF_SCENARIO_ENV]: providerScenarioId,
- };
-}
-
export class PerfWsRpcClient {
private readonly runtime: ManagedRuntime.ManagedRuntime<RpcClient.Protocol, never>;
private readonly clientScope: Scope.Closeable;
diff --git a/apps/web/test/perf/appHarness.ts b/apps/web/test/perf/appHarness.ts
--- a/apps/web/test/perf/appHarness.ts
+++ b/apps/web/test/perf/appHarness.ts
@@ -266,7 +266,7 @@
seedProcess.stderr?.on("data", (chunk) => {
stderr += chunk.toString();
});
- const [exitCode] = (await once(seedProcess, "exit")) as [number | null];
+ const [exitCode] = (await once(seedProcess, "close")) as [number | null];
if (exitCode !== 0) {
throw new Error(`Perf seed command failed with code ${exitCode ?? "unknown"}.\n${stderr}`);
}
diff --git a/apps/web/test/perf/serverEnv.ts b/apps/web/test/perf/serverEnv.ts
--- a/apps/web/test/perf/serverEnv.ts
+++ b/apps/web/test/perf/serverEnv.ts
@@ -1,27 +1,5 @@
-import type { PerfProviderScenarioId } from "@t3tools/shared/perf/scenarioCatalog";
-
-export const PERF_PROVIDER_ENV = "T3CODE_PERF_PROVIDER";
-export const PERF_SCENARIO_ENV = "T3CODE_PERF_SCENARIO";
-const AUTO_BOOTSTRAP_PROJECT_ENV = "T3CODE_AUTO_BOOTSTRAP_PROJECT_FROM_CWD";
-
-export function buildPerfServerEnv(
- baseEnv: NodeJS.ProcessEnv,
- providerScenarioId?: PerfProviderScenarioId,
-): NodeJS.ProcessEnv {
- const env: NodeJS.ProcessEnv = {
- ...baseEnv,
- [AUTO_BOOTSTRAP_PROJECT_ENV]: "false",
- };
-
- if (!providerScenarioId) {
- delete env[PERF_PROVIDER_ENV];
- delete env[PERF_SCENARIO_ENV];
- return env;
- }
-
- return {
- ...env,
- [PERF_PROVIDER_ENV]: "1",
- [PERF_SCENARIO_ENV]: providerScenarioId,
- };
-}
+export {
+ buildPerfServerEnv,
+ PERF_PROVIDER_ENV,
+ PERF_SCENARIO_ENV,
+} from "@t3tools/shared/perf/serverEnv";
diff --git a/packages/shared/package.json b/packages/shared/package.json
--- a/packages/shared/package.json
+++ b/packages/shared/package.json
@@ -51,6 +51,10 @@
"./perf/artifact": {
"types": "./src/perf/artifact.ts",
"import": "./src/perf/artifact.ts"
+ },
+ "./perf/serverEnv": {
+ "types": "./src/perf/serverEnv.ts",
+ "import": "./src/perf/serverEnv.ts"
}
},
"scripts": {
diff --git a/packages/shared/src/perf/serverEnv.ts b/packages/shared/src/perf/serverEnv.ts
new file mode 100644
--- /dev/null
+++ b/packages/shared/src/perf/serverEnv.ts
@@ -1,0 +1,27 @@
+import type { PerfProviderScenarioId } from "./scenarioCatalog.ts";
+
+export const PERF_PROVIDER_ENV = "T3CODE_PERF_PROVIDER";
+export const PERF_SCENARIO_ENV = "T3CODE_PERF_SCENARIO";
+const AUTO_BOOTSTRAP_PROJECT_ENV = "T3CODE_AUTO_BOOTSTRAP_PROJECT_FROM_CWD";
+
+export function buildPerfServerEnv(
+ baseEnv: NodeJS.ProcessEnv,
+ providerScenarioId?: PerfProviderScenarioId,
+): NodeJS.ProcessEnv {
+ const env: NodeJS.ProcessEnv = {
+ ...baseEnv,
+ [AUTO_BOOTSTRAP_PROJECT_ENV]: "false",
+ };
+
+ if (!providerScenarioId) {
+ delete env[PERF_PROVIDER_ENV];
+ delete env[PERF_SCENARIO_ENV];
+ return env;
+ }
+
+ return {
+ ...env,
+ [PERF_PROVIDER_ENV]: "1",
+ [PERF_SCENARIO_ENV]: providerScenarioId,
+ };
+}
diff --git a/scripts/open-perf-app.ts b/scripts/open-perf-app.ts
--- a/scripts/open-perf-app.ts
+++ b/scripts/open-perf-app.ts
@@ -208,7 +208,7 @@
stderr += chunk.toString();
});
- const [exitCode] = (await once(seedProcess, "exit")) as [number | null];
+ const [exitCode] = (await once(seedProcess, "close")) as [number | null];
if (exitCode !== 0) {
throw new Error(`Perf seed command failed with code ${exitCode ?? "unknown"}.\n${stderr}`);
}You can send follow-ups to this agent here.
| seedProcess.stderr?.on("data", (chunk) => { | ||
| stderr += chunk.toString(); | ||
| }); | ||
| const [exitCode] = (await once(seedProcess, "exit")) as [number | null]; |
Contributor
There was a problem hiding this comment.
Seed process stdout may be incomplete due to 'exit' event
Medium Severity
The code awaits once(seedProcess, "exit") but the child process uses piped stdio. The 'exit' event fires when the process terminates, but pipe buffers may not have fully drained yet — remaining 'data' events can arrive after the 'exit' promise resolves. This means stdout may be incomplete when parsePerfSeededState(stdout) runs, causing JSON parse failures or missing marker errors. Using once(seedProcess, "close") instead ensures all piped data has been consumed before proceeding.
Additional Locations (1)
| [PERF_PROVIDER_ENV]: "1", | ||
| [PERF_SCENARIO_ENV]: providerScenarioId, | ||
| }; | ||
| } |
Contributor
There was a problem hiding this comment.
Duplicated buildPerfServerEnv across server and web harnesses
Low Severity
buildPerfServerEnv is identically implemented in both serverPerfHarness.ts (private) and serverEnv.ts (exported). This function contains non-trivial env var deletion and conditional provider-enablement logic. Since the shared perf package already exists at @t3tools/shared/perf/, this utility could live there to avoid the two copies diverging when the env var contract changes.
Additional Locations (1)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing
PerfProviderAdapterevent emission.Note
Medium Risk
Mostly adds new perf-only seeding, provider playback, and Playwright/Vitest harnesses, but it also changes server/provider wiring behind env flags and adjusts several
Streamfields to getters, which could affect runtime behavior if misused.Overview
Adds a local performance regression harness that can seed deterministic server state, run the built server + built web app, drive the UI with Playwright, and write JSON + log artifacts under
artifacts/.Introduces a shared perf scenario catalog and artifact helpers (
@t3tools/shared/perf/*), plus server/web perf suites: server-side websocket/command/git latency benchmarks and web-side virtualization + websocket application benchmarks with dedicatedtest:perfentrypoints/config.Adds a perf-only provider implementation (
PerfProviderAdapter+ perfProviderRegistry) that replays paced runtime events and is selected at startup whenT3CODE_PERF_PROVIDER=1, along with aseedPerfStateworkflow/CLI and small testability hooks (e.g.,data-testidonChatView).Written by Cursor Bugbot for commit 0be82d3. This will update automatically on new commits. Configure here.
Note
Add performance regression test harnesses for server and web app
seedPerfStateto build reusable seeded SQLite state from two scenarios (large_threads,burst_base) by projecting synthetic events into a snapshot, then copying the template into a fresh run directory per test.startServerPerfHarnessin serverPerfHarness.ts to spawn the server against seeded state, connect via WebSocket RPC, and collect latency artifacts.startPerfAppHarnessin appHarness.ts to boot the built web server and a headless Chromium browser, instrument RAF/long-task metrics, time UI actions, and write JSON artifacts.PerfProviderAdapterandPerfProviderRegistryLiveto simulate thecodexprovider in-process, emitting deterministic timed events for thedense_assistant_streamscenario without real network calls.streamEventsfrom a static property to a getter on all provider adapters and related test harnesses, so each access returns a freshStreaminstance.perf:openandtest:perfscripts to the root and apppackage.jsonfiles for running or manually exploring perf scenarios.Macroscope summarized 0be82d3.