Extend the benchmark with a pathological meta-schema test

[jviotti]

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

[augmentcode]

🤖 Augment PR Summary

Summary: Extends the benchmark suite to cover more realistic and pathological indexer workloads, including a custom meta-schema stress case.

Changes:

• Installs jq in the benchmark image and copies additional benchmark scripts into the container.
• Splits the previous monolithic benchmark into dedicated scripts for add/update/cached-rebuild scenarios, and for “index N schemas” cold runs.
• Adds a new benchmark that indexes many schemas using a user-defined custom 2020-12 meta-schema to stress resolver/meta-schema handling.
• Updates benchmark/index.sh to orchestrate all sub-benchmarks and merge their JSON outputs via jq -s 'add'.

Technical Notes: Benchmark scripts emit JSON arrays on stdout (for aggregation) while printing progress to stderr; Dockerfile now ensures jq is available for the aggregation step.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

[cubic-dev-ai]

4 issues found across 5 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[github-actions]

Benchmark Index (community)

Details

Benchmark suite	Current: a50f7ca	Previous: 3107c27	Ratio
Add one schema (0 existing)	392 ms	409 ms	0.96
Add one schema (100 existing)	28 ms	30 ms	0.93
Add one schema (1000 existing)	84 ms	91 ms	0.92
Add one schema (10000 existing)	774 ms	740 ms	1.05
Update one schema (1 existing)	22 ms	22 ms	1
Update one schema (101 existing)	29 ms	31 ms	0.94
Update one schema (1001 existing)	87 ms	91 ms	0.96
Update one schema (10001 existing)	737 ms	749 ms	0.98
Cached rebuild (1 existing)	6 ms	7 ms	0.86
Cached rebuild (101 existing)	8 ms	9 ms	0.89
Cached rebuild (1001 existing)	29 ms	31 ms	0.94
Cached rebuild (10001 existing)	250 ms	273 ms	0.92
Index 100 schemas	638 ms	662 ms	0.96
Index 1000 schemas	1629 ms	1574 ms	1.03
Index 10000 schemas	13715 ms	13522 ms	1.01
Index 10000 schemas (custom meta-schema)	147137 ms

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark Index (enterprise)

Details

Benchmark suite	Current: a50f7ca	Previous: 3107c27	Ratio
Add one schema (0 existing)	407 ms	402 ms	1.01
Add one schema (100 existing)	34 ms	32 ms	1.06
Add one schema (1000 existing)	92 ms	88 ms	1.05
Add one schema (10000 existing)	813 ms	691 ms	1.18
Update one schema (1 existing)	24 ms	26 ms	0.92
Update one schema (101 existing)	33 ms	31 ms	1.06
Update one schema (1001 existing)	91 ms	89 ms	1.02
Update one schema (10001 existing)	699 ms	699 ms	1
Cached rebuild (1 existing)	8 ms	7 ms	1.14
Cached rebuild (101 existing)	10 ms	9 ms	1.11
Cached rebuild (1001 existing)	32 ms	29 ms	1.10
Cached rebuild (10001 existing)	240 ms	247 ms	0.97
Index 100 schemas	621 ms	636 ms	0.98
Index 1000 schemas	1626 ms	1536 ms	1.06
Index 10000 schemas	13716 ms	13838 ms	0.99
Index 10000 schemas (custom meta-schema)	144430 ms

This comment was automatically generated by workflow using github-action-benchmark.