Feat: Drop report.txt and use results_summary.json

[nvzhihanj]

Narrowed scope: this PR now covers only the result_summary.json / report.txt change.

What & why

result_summary.json and report.txt are two renderings of the same Report. report.txt has no programmatic consumers, and result_summary.json was missing the computed QPS/TPS (callers had to recompute them from duration + counts).

Changes

• result_summary.json is now self-complete. Report.to_json injects the derived qps/tps into the serialized JSON. They remain methods (not stored fields) to avoid duplicating data already in duration_ns / counters / OSL, so no Report constructor sites change.
• report.txt is no longer written. It was a human-readable dump of the same Report with zero programmatic/test consumers; the same summary is still logged to the console.
• Docs: docs/LOCAL_TESTING.md — result_summary.json described as the primary, self-complete report; dropped report.txt and the never-written runtime_settings.json from the artifacts list.

Tests

• result_summary.json serializes qps/tps (concrete value) and serializes them as null when there is no duration.
• End-to-end perf run: result_summary.json is self-complete and no report.txt is produced.
• Full suite + pre-commit green.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Code Review

This pull request introduces a new truncate-results CLI command to shrink large benchmark results files by keeping a few full responses and hashing the rest. It also optimizes artifact generation by only running the event logger and saving event logs or sample index maps when accuracy scoring is active or when explicitly requested via configuration flags. Additionally, derived throughput metrics (qps and tps) are now directly serialized into the JSON report. The review feedback suggests several robust improvements: implementing atomic file writes and better error handling in the truncation command, wrapping the tmpfs salvage process in a try-except block to ensure resource cleanup, and using msgspec.structs.asdict for a more efficient struct-to-dict conversion.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Writing directly to the target file is risky, especially for large files (which can be gigabytes in size). If the write operation is interrupted or fails due to disk space issues, the original file will be corrupted or lost. Writing to a temporary file first and then atomically replacing the target file prevents corruption.

    if config.in_place:
        out_path = config.results
    elif config.output is not None:
        out_path = config.output
    else:
        out_path = config.results.with_name(config.results.stem + ".truncated.json")

    temp_path = out_path.with_suffix(".tmp")
    try:
        temp_path.write_text(json.dumps(truncated, indent=2))
        temp_path.replace(out_path)
    except Exception as e:
        if temp_path.exists():
            temp_path.unlink()
        raise e

[gemini-code-assist]

If _salvage_tmpfs raises an exception (e.g., due to a destination disk-full or permission error during shutil.copy2), the subsequent shutil.rmtree call will be skipped. This will leak the multi-gigabyte tmpfs_dir in RAM (/dev/shm), potentially leading to RAM exhaustion on subsequent runs. Wrapping the salvage call in a try-except block ensures cleanup always runs.

Suggested change

	_salvage_tmpfs(
	ctx.report_dir,
	bench.tmpfs_dir,
	keep_events=ctx.config.keep_events_artifacts,
	)
	try:
	_salvage_tmpfs(
	ctx.report_dir,
	bench.tmpfs_dir,
	keep_events=ctx.config.keep_events_artifacts,
	)
	except Exception as e:
	logger.warning(f"Failed to salvage tmpfs: {e}")

[gemini-code-assist]

Using is_file() is safer than exists() because it ensures the path is a regular file and not a directory, preventing potential IsADirectoryError exceptions when calling read_text().

Suggested change

	if not config.results.exists():
	raise InputValidationError(f"Results file not found: {config.results}")
	if not config.results.is_file():
	raise InputValidationError(f"Results file not found or is not a file: {config.results}")

[gemini-code-assist]

If the results file is empty or contains malformed JSON, json.loads will raise a json.JSONDecodeError, which propagates as an unhandled exception and prints a full traceback. Handling this gracefully and raising an InputValidationError provides a much better user experience.

    try:
        data = json.loads(config.results.read_text())
    except json.JSONDecodeError as e:
        raise InputValidationError(f"Invalid JSON in results file: {e}") from e

[gemini-code-assist]

Using msgspec.json.decode(msgspec.json.encode(self)) to convert a msgspec.Struct to a dictionary introduces unnecessary serialization and deserialization overhead. Using msgspec.structs.asdict is the standard, high-performance way to convert a struct to a dictionary.

Suggested change

	payload: dict[str, Any] = msgspec.json.decode(msgspec.json.encode(self))
	import msgspec
	payload = msgspec.structs.asdict(self)