From 98067b68e1c90ab223286787c24f7cff16ab322c Mon Sep 17 00:00:00 2001 From: Yoshifumi Nakamura Date: Tue, 16 Jun 2026 11:53:30 +0900 Subject: [PATCH] Clarify result server key roles and shell CI deps Assert jq and python3 in the lightweight Result Server shell-test job so estimation and profile-data tests cannot silently skip because their runtime dependencies are absent. Document RESULT_SERVER_KEYS as the server-side registry of accepted posting/query keys for future multi-source ingestion, while each CI job or client still receives a single RESULT_SERVER_KEY secret for its own upload/query operations. Validation: WSL jq 1.6, shellcheck 0.8.0, python3.12 and python3 available; test_bk_profiler.sh, test_result_profile_data.sh, test_send_results_profile_data.sh, test_send_estimate_artifacts.sh, test_estimation_gpu_kernel_mlp_v15.sh, test_genesis_gpu_mlp_estimation.sh, and test_qws_gpu_mlp_smoke_estimation.sh passed. Signed-off-by: Yoshifumi Nakamura --- .github/workflows/result-server-tests.yml | 8 ++++++++ docs/deploy/key-management.md | 19 ++++++++++++++++--- docs/guides/developer-reference.md | 4 +++- 3 files changed, 27 insertions(+), 4 deletions(-) diff --git a/.github/workflows/result-server-tests.yml b/.github/workflows/result-server-tests.yml index 8a91c87..137f601 100644 --- a/.github/workflows/result-server-tests.yml +++ b/.github/workflows/result-server-tests.yml @@ -88,6 +88,14 @@ jobs: - name: Check out repository uses: actions/checkout@v4 + - name: Check shell test dependencies + run: | + set -euo pipefail + command -v jq + jq --version + command -v python3 + python3 --version + - name: Run profiler, profile-data, and estimation shell tests run: | bash scripts/tests/test_bk_profiler.sh diff --git a/docs/deploy/key-management.md b/docs/deploy/key-management.md index 81b9616..dd18c11 100644 --- a/docs/deploy/key-management.md +++ b/docs/deploy/key-management.md @@ -9,12 +9,24 @@ Production deployments must provide: - `FLASK_SECRET_KEY`: at least 32 characters, generated randomly. - `RESULT_SERVER_KEYS`: one or more runner-scoped ingest keys. -Use runner-scoped keys instead of the legacy `RESULT_SERVER_KEY`: +Use runner-scoped server keys instead of the legacy server-side +`RESULT_SERVER_KEY` fallback: ```text RESULT_SERVER_KEYS=runner-a:,runner-b: ``` +`RESULT_SERVER_KEYS` is the server-side registry of accepted posting/query +keys. It is intentionally broader than the current single-key CI setup so that +the portal can later accept results from multiple trusted CI sources, such as +the main BenchKit CI, site-managed runners, collaborator forks, or +estimator-only pipelines. + +Each client job still receives a single `RESULT_SERVER_KEY` secret for its own +uploads. This client-side key must match one entry in `RESULT_SERVER_KEYS`, and +it is typically injected through GitLab CI/CD variables or another CI secret +mechanism rather than stored on the runner host. + Each key must be at least 32 characters and must not use known insecure examples such as `dev-api-key`, `changeme`, or `secret`. The production app refuses to start when these checks fail. @@ -37,12 +49,13 @@ For a normal runner key rotation: 1. Add the new key to `RESULT_SERVER_KEYS` while keeping the old key. 2. Deploy the portal configuration. -3. Update the runner to use the new key. +3. Update the corresponding CI secret so affected jobs receive the new + client-side `RESULT_SERVER_KEY`. 4. Confirm successful ingest events for the runner. 5. Remove the old key after the agreed overlap window. If a key may have leaked, remove it immediately, deploy the portal, update the -affected runner, and review ingest logs for suspicious activity. +affected CI secret, and review ingest logs for suspicious activity. ## Logging diff --git a/docs/guides/developer-reference.md b/docs/guides/developer-reference.md index 9c968a2..1fb8c81 100644 --- a/docs/guides/developer-reference.md +++ b/docs/guides/developer-reference.md @@ -248,8 +248,10 @@ For production portal deployments: - Set `FLASK_SECRET_KEY` to a strong secret and run `result_server/app.py`, not `app_dev.py`. - `app.py` binds to `127.0.0.1:8800` by default; set `RESULT_SERVER_HOST` and `RESULT_SERVER_PORT` explicitly when the deployment requires a different bind address. - Set runner-scoped ingest keys with `RESULT_SERVER_KEYS=runner-a:,runner-b:`. +- `RESULT_SERVER_KEYS` is the server-side registry of accepted posting/query keys. It is intentionally broader than the current single-key CI setup so the portal can later accept results from multiple trusted CI sources such as main BenchKit CI, site-managed runners, collaborator forks, or estimator-only pipelines. +- Each client job still receives a single `RESULT_SERVER_KEY` secret for its own upload/query operations, usually through GitLab CI/CD variables or another CI secret store. That client-side key must match one entry in server-side `RESULT_SERVER_KEYS`. - `FLASK_SECRET_KEY` and each ingest key must be at least 32 characters and must not use known insecure examples such as `dev-api-key`, `changeme`, or `secret`; production startup refuses these values. -- The legacy `RESULT_SERVER_KEY` variable is still accepted as runner `default` for compatibility, but should be rotated to `RESULT_SERVER_KEYS`. +- The legacy server-side `RESULT_SERVER_KEY` variable is still accepted as runner `default` for compatibility, but production portal deployments should rotate the accepted-key registry to `RESULT_SERVER_KEYS`. - See `docs/deploy/key-management.md` for generation and rotation guidance. - `REDIS_URL` must point to a monitored Redis instance; production authentication refuses login when Redis is unavailable. - Login verification, API ingest/query, and admin write endpoints use Redis-backed rate limits by default; set `RESULT_SERVER_MAX_UPLOAD_MB` and `RESULT_SERVER_MAX_ARCHIVE_MEMBER_MB` when deployment-specific upload limits are needed.