FogHTTP Benchmark

Reproducible benchmark suite for FogHTTP and comparable Python HTTP clients. The benchmark installs foghttp from PyPI and treats it as an external user-facing dependency.

The goal is not marketing-perfect numbers. The goal is a repeatable, readable harness that shows real trade-offs across buffered HTTP workloads, redirects, pool contention, delay scenarios, resource usage, and client lifecycle cost. For FogHTTP 0.3.x, the harness also includes a dedicated resource/backpressure suite for active request limits, per-origin limits, pending queues, pool timeouts, and buffered response body limits, plus a one-upstream API client suite for base_url, client defaults, params merging, prepared requests, and request body encoding. It also includes a request builder suite for Python-side build_request() and query construction cost, and a compressed-response suite for transparent gzip, deflate, and br decode overhead.

What It Measures

• async and sync buffered request throughput
• latency percentiles per scenario
• redirect behavior for GET, HEAD, and POST
• local pool contention and delayed responses
• peak RSS, thread count, and file descriptor pressure
• client creation, first request, reuse, and close cost
• one-upstream API client overhead for defaults and prepared requests
• pure request builder overhead before network I/O
• transparent compressed response decode overhead
• aggregate buffered response budget behavior

Install

uv sync

The project dependency on foghttp is resolved from PyPI:

"foghttp>=0.3,<0.4"

To benchmark a different released version, change the dependency constraint and refresh the lock file. Avoid benchmarking a local editable checkout unless the explicit goal is pre-release development analysis.

Request Benchmark

uv run foghttp-benchmark \
  --clients foghttp,httpx,aiohttp,zapros \
  --modes async,sync \
  --requests 500 \
  --warmup 50 \
  --repeats 3 \
  --concurrency 1,10,50,100 \
  --scenarios json-small,json-decode-small,bytes-64k,post-json-echo,post-echo-64k,redirect-get-302,redirect-head-302,redirect-post-303,redirect-post-307 \
  --output-dir results/local

Client Creation Benchmark

uv run foghttp-benchmark \
  --suite client-creation \
  --clients foghttp,httpx,aiohttp,zapros \
  --modes async,sync \
  --iterations 100 \
  --repeats 3 \
  --client-counts 1,10,50 \
  --output-dir results/client-creation

Resource / Backpressure Benchmark

uv run foghttp-benchmark \
  --suite resource-backpressure \
  --clients foghttp \
  --modes async,sync \
  --requests 200 \
  --warmup 0 \
  --repeats 3 \
  --concurrency 10,50,100 \
  --output-dir results/resource-backpressure

This suite is FogHTTP-specific because it uses FogHTTP public transport diagnostics. It checks global active request slots, per-origin active request slots, bounded pending requests, PoolTimeout behavior, recovery after timeout bursts, max_response_body_size cleanup, and max_buffered_response_bytes aggregate budget behavior.

Compressed Response Benchmark

uv run foghttp-benchmark \
  --suite compressed-response \
  --clients foghttp,httpx,aiohttp,zapros \
  --modes async,sync \
  --requests 1000 \
  --warmup 100 \
  --repeats 3 \
  --concurrency 1,10,50 \
  --output-dir results/compressed-response

This suite measures transparent decode overhead for buffered compressed responses: small JSON, 64 KiB bodies, a high-ratio 1 MiB body, and a multi-field Content-Encoding response. It is useful for checking the cost of FogHTTP 0.3.1 response decoding against clients that already perform automatic decompression.

One-Upstream API Client Benchmark

uv run foghttp-benchmark \
  --suite one-upstream \
  --clients foghttp,httpx,aiohttp,zapros \
  --modes async,sync \
  --requests 1000 \
  --warmup 100 \
  --repeats 3 \
  --concurrency 1,10,50 \
  --output-dir results/one-upstream

This suite compares foghttp and httpx in the common service-client pattern: one upstream per client, base_url, default headers, default params, per-request params, JSON/form bodies, and prepared requests. Clients without a semantics-compatible defaults API are reported as skipped.

Request Builder Benchmark

uv run foghttp-benchmark \
  --suite request-builder \
  --clients foghttp,httpx,aiohttp,zapros \
  --modes async,sync \
  --iterations 5000 \
  --warmup 500 \
  --repeats 3 \
  --output-dir results/request-builder

This suite measures Python-side build_request() cost separately from network I/O. Pure build cases do not start a server. The send-prepared-get case starts the local loopback server and measures the combined build-plus-send path through a reused client. Unsupported clients are reported as skipped.

Outputs

Each run writes timestamped JSON and Markdown reports plus latest.json and latest.md links/copies in the selected output directory. Generated results are ignored by git by default. Publish selected reports intentionally when they are part of a release or benchmark note.

Compare Reports

Use compare to turn two JSON reports from the same suite into a compact Markdown delta report:

uv run foghttp-benchmark compare \
  results/full-requests/latest.json \
  results/full-requests-0.2.1/latest.json \
  --output results/compare-requests.md

The comparison highlights geomean and median ratios, competitive wins, per-mode/per-scenario deltas, top improvements, top regressions, error rows, resource peaks, and unmatched focus rows.

Progress Output

Benchmark runs show stages and completed run counts by default. Use --no-progress for quiet machine-readable runs:

uv run foghttp-benchmark --no-progress --output-dir results/local

Interactive terminals use Rich progress bars. Plain log output keeps outer suite milestones, while inner request-load progress is emitted as a heartbeat only for long-running stages.

Methodology

• Benchmarks use a local asyncio HTTP/1.1 loopback server.
• Scenarios are shuffled by default with a stable seed.
• Sync and async results should be compared separately.
• In request reports, limit means the configured benchmark pressure limit. For FogHTTP 0.2.x it maps to active request slots and idle pool capacity; for other clients it maps to their connection pool limit.
• Higher ok/s or lifecycle ops/s is better.
• Lower latency, thread count, file descriptor count, memory delta, and errors are better.
• Local loopback results do not measure real internet latency, DNS behavior, TLS handshake cost, HTTP/2, proxies, cookies, streaming, or auth flows.

Development

uv sync --extra dev
uv run python -m py_compile foghttp_benchmark/*.py foghttp_benchmark/clients/*.py foghttp_benchmark/creation/*.py
uv run --extra dev ruff format .
uv run --extra dev ruff check .
uv run --extra dev mypy
pre-commit run --all-files

Keep benchmark code decomposed. If a scenario grows into a subsystem, split it into modules before it becomes difficult to review.