Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

## Table of Contents

- [What's new (2026-06-22) — Time-Series Transforms](#whats-new-2026-06-22--time-series-transforms)
- [What's new (2026-06-22) — Unicode Text Normalisation & Slugify](#whats-new-2026-06-22--unicode-text-normalisation--slugify)
- [What's new (2026-06-22) — JSON-Schema Compatibility Checking](#whats-new-2026-06-22--json-schema-compatibility-checking)
- [What's new (2026-06-22) — Typed Configuration Schema](#whats-new-2026-06-22--typed-configuration-schema)
Expand Down Expand Up @@ -150,6 +151,12 @@

---

## What's new (2026-06-22) — Time-Series Transforms

Turn counters into rates; downsample and resample. Full reference: [`docs/source/Eng/doc/new_features/v98_features_doc.rst`](docs/source/Eng/doc/new_features/v98_features_doc.rst).

- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`** (`AC_ts_rate`, `AC_ts_downsample`): `observability` counters store only the current value (no counter→rate anywhere) and `cost_telemetry` only buckets by day. This adds Prometheus-style reset-aware rate/increase/delta over `(timestamp, value)` series, tumbling-bucket downsampling (avg/sum/min/max/first/last/count), and grid resampling (last/linear/none). No wall clock — deterministic. Pure-stdlib.

## What's new (2026-06-22) — Unicode Text Normalisation & Slugify

Canonicalize text before fuzzy/search/OCR matching. Full reference: [`docs/source/Eng/doc/new_features/v97_features_doc.rst`](docs/source/Eng/doc/new_features/v97_features_doc.rst).
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目录

- [本次更新 (2026-06-22) — 时间序列变换](#本次更新-2026-06-22--时间序列变换)
- [本次更新 (2026-06-22) — Unicode 文本规范化与 Slug](#本次更新-2026-06-22--unicode-文本规范化与-slug)
- [本次更新 (2026-06-22) — JSON-Schema 兼容性检查](#本次更新-2026-06-22--json-schema-兼容性检查)
- [本次更新 (2026-06-22) — 具类型的配置结构](#本次更新-2026-06-22--具类型的配置结构)
Expand Down Expand Up @@ -149,6 +150,12 @@

---

## 本次更新 (2026-06-22) — 时间序列变换

把计数器转成速率;降采样与重采样。完整参考:[`docs/source/Zh/doc/new_features/v98_features_doc.rst`](../docs/source/Zh/doc/new_features/v98_features_doc.rst)。

- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`**(`AC_ts_rate`、`AC_ts_downsample`):`observability` 计数器只存当前值(无处可把计数器转速率),`cost_telemetry` 只以天分桶。本功能在 `(timestamp, value)` 序列上加入 Prometheus 风格、具重置感知的 rate/increase/delta、tumbling-bucket 降采样(avg/sum/min/max/first/last/count)与网格重采样(last/linear/none)。不读 wall clock、确定。纯标准库。

## 本次更新 (2026-06-22) — Unicode 文本规范化与 Slug

在 fuzzy/search/OCR 匹配前规范化文本。完整参考:[`docs/source/Zh/doc/new_features/v97_features_doc.rst`](../docs/source/Zh/doc/new_features/v97_features_doc.rst)。
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目錄

- [本次更新 (2026-06-22) — 時間序列轉換](#本次更新-2026-06-22--時間序列轉換)
- [本次更新 (2026-06-22) — Unicode 文字正規化與 Slug](#本次更新-2026-06-22--unicode-文字正規化與-slug)
- [本次更新 (2026-06-22) — JSON-Schema 相容性檢查](#本次更新-2026-06-22--json-schema-相容性檢查)
- [本次更新 (2026-06-22) — 具型別的設定結構](#本次更新-2026-06-22--具型別的設定結構)
Expand Down Expand Up @@ -149,6 +150,12 @@

---

## 本次更新 (2026-06-22) — 時間序列轉換

把計數器轉成速率;降採樣與重採樣。完整參考:[`docs/source/Zh/doc/new_features/v98_features_doc.rst`](../docs/source/Zh/doc/new_features/v98_features_doc.rst)。

- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`**(`AC_ts_rate`、`AC_ts_downsample`):`observability` 計數器只存當前值(無處可把計數器轉速率),`cost_telemetry` 只以天分桶。本功能在 `(timestamp, value)` 序列上加入 Prometheus 風格、具重置感知的 rate/increase/delta、tumbling-bucket 降採樣(avg/sum/min/max/first/last/count)與網格重採樣(last/linear/none)。不讀 wall clock、具決定性。純標準函式庫。

## 本次更新 (2026-06-22) — Unicode 文字正規化與 Slug

在 fuzzy/search/OCR 比對前正規化文字。完整參考:[`docs/source/Zh/doc/new_features/v97_features_doc.rst`](../docs/source/Zh/doc/new_features/v97_features_doc.rst)。
Expand Down
43 changes: 43 additions & 0 deletions docs/source/Eng/doc/new_features/v98_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Time-Series Transforms
======================

``observability`` counters and gauges store only the *current* value — nothing
turned a counter into a per-second rate — and ``cost_telemetry`` only buckets by
a fixed day. This adds Prometheus-style ``rate`` / ``irate`` / ``increase`` /
``delta`` (reset-aware) plus tumbling-bucket ``downsample`` and grid
``resample`` over ``(timestamp, value)`` sequences.

Pure standard library (``bisect``); imports no ``PySide6``. No wall clock is
read — windows use the series' own timestamps — so every function is fully
deterministic in CI.

Headless API
------------

.. code-block:: python

from je_auto_control import ts_rate, ts_increase, ts_downsample, ts_resample

series = [(0, 0), (10, 50), (20, 120)] # (timestamp_s, counter_value)
ts_rate(series) # 6.0 (120 over 20s)
ts_rate(series, window_s=10) # rate over the last 10s only
ts_increase(series) # 120.0 (reset-aware)

ts_downsample([(0, 1), (3, 3), (5, 10)], 5, "avg") # [(0, 2.0), (5, 10.0)]
ts_resample([(0, 0), (20, 20)], 10, fill="linear") # [(0,0),(10,10),(20,20)]

``ts_rate`` / ``ts_increase`` treat a value drop as a counter reset (Prometheus
semantics); ``ts_irate`` is the instant rate from the last two samples;
``ts_delta`` / ``ts_idelta`` are gauge first-to-last and last-two differences.
``ts_downsample`` rolls the series into ``bucket_s`` tumbling buckets aggregated
by ``avg`` / ``sum`` / ``min`` / ``max`` / ``first`` / ``last`` / ``count``.
``ts_resample`` aligns to a fixed grid, filling with ``"last"`` (carry forward),
``"linear"`` (interpolate), or ``None`` (gaps).

Executor commands
-----------------

``AC_ts_rate`` returns ``{rate}`` for a ``series`` (optional ``window_s``);
``AC_ts_downsample`` returns ``{buckets}`` for a ``series`` and ``bucket_s``
(optional ``agg``). Both are exposed as MCP tools (``ac_ts_rate`` /
``ac_ts_downsample``) and as Script Builder commands under **Data**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v95_features_doc
doc/new_features/v96_features_doc
doc/new_features/v97_features_doc
doc/new_features/v98_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
37 changes: 37 additions & 0 deletions docs/source/Zh/doc/new_features/v98_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
時間序列轉換
==========

``observability`` 的計數器與量規只儲存*當前*值 —— 沒有任何東西能把計數器轉成每秒速率 —— 而
``cost_telemetry`` 只以固定的「天」分桶。本功能在 ``(timestamp, value)`` 序列上加入 Prometheus 風格的
``rate`` / ``irate`` / ``increase`` / ``delta``(具重置感知),以及 tumbling-bucket ``downsample`` 與
網格 ``resample``。

純標準函式庫(``bisect``);不匯入 ``PySide6``。不讀取 wall clock —— 視窗使用序列自身的時間戳 —— 因此每個
函式皆完全具決定性。

無頭 API
--------

.. code-block:: python

from je_auto_control import ts_rate, ts_increase, ts_downsample, ts_resample

series = [(0, 0), (10, 50), (20, 120)] # (timestamp_s, counter_value)
ts_rate(series) # 6.0(20 秒內 120)
ts_rate(series, window_s=10) # 只看最後 10 秒的速率
ts_increase(series) # 120.0(重置感知)

ts_downsample([(0, 1), (3, 3), (5, 10)], 5, "avg") # [(0, 2.0), (5, 10.0)]
ts_resample([(0, 0), (20, 20)], 10, fill="linear") # [(0,0),(10,10),(20,20)]

``ts_rate`` / ``ts_increase`` 把值下降視為計數器重置(Prometheus 語意);``ts_irate`` 是最後兩個樣本的
瞬時速率;``ts_delta`` / ``ts_idelta`` 是量規的首尾差與最後兩點差。``ts_downsample`` 把序列滾成 ``bucket_s``
的 tumbling 桶,以 ``avg`` / ``sum`` / ``min`` / ``max`` / ``first`` / ``last`` / ``count`` 聚合。
``ts_resample`` 對齊到固定網格,以 ``"last"``(前向填補)、``"linear"``(內插)或 ``None``(留缺)填值。

執行器命令
----------

``AC_ts_rate`` 對 ``series``(可選 ``window_s``)回傳 ``{rate}``;``AC_ts_downsample`` 對 ``series`` 與
``bucket_s``(可選 ``agg``)回傳 ``{buckets}``。兩者皆以 MCP 工具(``ac_ts_rate`` / ``ac_ts_downsample``)
以及 Script Builder 中 **Data** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v95_features_doc
doc/new_features/v96_features_doc
doc/new_features/v97_features_doc
doc/new_features/v98_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
7 changes: 7 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,11 @@
)
# Mergeable streaming latency digest + exact percentiles
from je_auto_control.utils.percentiles import LatencyDigest, exact_percentiles
# Time-series transforms: rate / irate / delta / downsample / resample
from je_auto_control.utils.timeseries import (
ts_delta, ts_downsample, ts_idelta, ts_increase, ts_irate, ts_rate,
ts_resample,
)
# Bulkhead concurrency isolation + rate-limit header parsing
from je_auto_control.utils.bulkhead import (
Bulkhead, BulkheadFullError, next_delay, parse_ratelimit, parse_retry_after,
Expand Down Expand Up @@ -978,6 +983,8 @@ def start_autocontrol_gui(*args, **kwargs):
"run_experiment",
"BurnRule", "burn_alerts", "burn_rate", "default_burn_rules", "evaluate_slo",
"LatencyDigest", "exact_percentiles",
"ts_delta", "ts_downsample", "ts_idelta", "ts_increase", "ts_irate",
"ts_rate", "ts_resample",
"Bulkhead", "BulkheadFullError", "next_delay", "parse_ratelimit",
"parse_retry_after",
"Cassette", "CassetteMissError",
Expand Down
20 changes: 20 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -1921,6 +1921,26 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
),
description="Classify JSON-Schema changes as backward/forward/full.",
))
specs.append(CommandSpec(
"AC_ts_rate", "Data", "Time-Series: Counter Rate",
fields=(
FieldSpec("series", FieldType.STRING,
placeholder="[[0, 0], [10, 50], [20, 120]]"),
FieldSpec("window_s", FieldType.FLOAT, optional=True),
),
description="Per-second counter rate (reset-aware) over a series.",
))
specs.append(CommandSpec(
"AC_ts_downsample", "Data", "Time-Series: Downsample",
fields=(
FieldSpec("series", FieldType.STRING,
placeholder="[[0, 1], [5, 3], [12, 9]]"),
FieldSpec("bucket_s", FieldType.FLOAT, placeholder="10"),
FieldSpec("agg", FieldType.STRING, optional=True,
placeholder="avg|sum|min|max|first|last|count"),
),
description="Roll a series into tumbling buckets by aggregate.",
))
specs.append(CommandSpec(
"AC_diff_rows", "Data", "Dataset Diff: Rows by Key",
fields=(
Expand Down
23 changes: 23 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -3369,6 +3369,27 @@ def _percentiles(samples: Any, qs: Any = None) -> Dict[str, Any]:
return {"percentiles": {str(q): value for q, value in result.items()}}


def _ts_rate(series: Any, window_s: Any = None) -> Dict[str, Any]:
"""Adapter: per-second counter rate over a (ts, value) series."""
import json
from je_auto_control.utils.timeseries import ts_rate
if isinstance(series, str):
series = json.loads(series)
window = float(window_s) if window_s is not None else None
return {"rate": ts_rate(series, window_s=window)}


def _ts_downsample(series: Any, bucket_s: Any,
agg: str = "avg") -> Dict[str, Any]:
"""Adapter: downsample a (ts, value) series into tumbling buckets."""
import json
from je_auto_control.utils.timeseries import ts_downsample
if isinstance(series, str):
series = json.loads(series)
buckets = ts_downsample(series, float(bucket_s), agg)
return {"buckets": [list(point) for point in buckets]}


def _evaluate_slo(records: Any, target: float,
window_s: Optional[float] = None) -> Dict[str, Any]:
"""Adapter: SLI + error budget for outcome records (list or JSON string)."""
Expand Down Expand Up @@ -4464,6 +4485,8 @@ def __init__(self):
"AC_resolve_config": _resolve_config,
"AC_explain_config": _explain_config,
"AC_check_compatibility": _check_compatibility,
"AC_ts_rate": _ts_rate,
"AC_ts_downsample": _ts_downsample,
"AC_detect_drift": _detect_drift,
"AC_categorical_drift": _categorical_drift,
"AC_diff_rows": _diff_rows,
Expand Down
29 changes: 29 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -3548,6 +3548,34 @@ def dataset_diff_tools() -> List[MCPTool]:
]


def timeseries_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_ts_rate",
description=("Per-second counter rate (reset-aware) over a 'series' "
"of [timestamp, value] pairs, optional 'window_s'. "
"Returns {rate}."),
input_schema=schema(
{"series": {"type": "array"}, "window_s": {"type": "number"}},
["series"]),
handler=h.ts_rate,
annotations=READ_ONLY,
),
MCPTool(
name="ac_ts_downsample",
description=("Roll a 'series' of [timestamp, value] pairs into "
"'bucket_s' tumbling buckets aggregated by 'agg' "
"(avg/sum/min/max/first/last/count). Returns {buckets}."),
input_schema=schema(
{"series": {"type": "array"}, "bucket_s": {"type": "number"},
"agg": {"type": "string"}},
["series", "bucket_s"]),
handler=h.ts_downsample,
annotations=READ_ONLY,
),
]


def schema_compat_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -5411,6 +5439,7 @@ def media_assert_tools() -> List[MCPTool]:
secret_ref_tools, config_schema_tools, config_redaction_tools,
data_profile_tools, http_problem_tools, dotenv_tools,
sse_client_tools, layered_config_tools, data_drift_tools, schema_compat_tools,
timeseries_tools,
dataset_diff_tools, referential_tools, link_header_tools, multipart_tools,
http_content_tools, cookie_jar_tools, http_conditional_tools,
saga_tools, decision_table_tools, locator_repair_tools,
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1880,6 +1880,16 @@ def check_compatibility(old, new, mode="backward"):
return _check_compatibility(old, new, mode)


def ts_rate(series, window_s=None):
from je_auto_control.utils.executor.action_executor import _ts_rate
return _ts_rate(series, window_s)


def ts_downsample(series, bucket_s, agg="avg"):
from je_auto_control.utils.executor.action_executor import _ts_downsample
return _ts_downsample(series, bucket_s, agg)


def detect_drift(reference, current, threshold=0.25, bins=10):
from je_auto_control.utils.executor.action_executor import _detect_drift
return _detect_drift(reference, current, threshold, bins)
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/timeseries/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""Time-series transforms (rate / downsample / resample) for AutoControl."""
from je_auto_control.utils.timeseries.timeseries import (
ts_delta, ts_downsample, ts_idelta, ts_increase, ts_irate, ts_rate,
ts_resample,
)

__all__ = [
"ts_delta", "ts_downsample", "ts_idelta", "ts_increase", "ts_irate",
"ts_rate", "ts_resample",
]
Loading
Loading