Skip to content

Commit 438decc

Browse files
JE-ChenJE-Chen
authored
Merge pull request #306 from Integration-Automation/feat/timeseries-batch
Add time-series transforms (rate / downsample / resample)
2 parents 331a6c2 + b4aa9ad commit 438decc

15 files changed

Lines changed: 431 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313

1414
## Table of Contents
1515

16+
- [What's new (2026-06-22) — Time-Series Transforms](#whats-new-2026-06-22--time-series-transforms)
1617
- [What's new (2026-06-22) — Unicode Text Normalisation & Slugify](#whats-new-2026-06-22--unicode-text-normalisation--slugify)
1718
- [What's new (2026-06-22) — JSON-Schema Compatibility Checking](#whats-new-2026-06-22--json-schema-compatibility-checking)
1819
- [What's new (2026-06-22) — Typed Configuration Schema](#whats-new-2026-06-22--typed-configuration-schema)
@@ -150,6 +151,12 @@
150151

151152
---
152153

154+
## What's new (2026-06-22) — Time-Series Transforms
155+
156+
Turn counters into rates; downsample and resample. Full reference: [`docs/source/Eng/doc/new_features/v98_features_doc.rst`](docs/source/Eng/doc/new_features/v98_features_doc.rst).
157+
158+
- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`** (`AC_ts_rate`, `AC_ts_downsample`): `observability` counters store only the current value (no counter→rate anywhere) and `cost_telemetry` only buckets by day. This adds Prometheus-style reset-aware rate/increase/delta over `(timestamp, value)` series, tumbling-bucket downsampling (avg/sum/min/max/first/last/count), and grid resampling (last/linear/none). No wall clock — deterministic. Pure-stdlib.
159+
153160
## What's new (2026-06-22) — Unicode Text Normalisation & Slugify
154161

155162
Canonicalize text before fuzzy/search/OCR matching. Full reference: [`docs/source/Eng/doc/new_features/v97_features_doc.rst`](docs/source/Eng/doc/new_features/v97_features_doc.rst).

‎README/README_zh-CN.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212

1313
## 目录
1414

15+
- [本次更新 (2026-06-22) — 时间序列变换](#本次更新-2026-06-22--时间序列变换)
1516
- [本次更新 (2026-06-22) — Unicode 文本规范化与 Slug](#本次更新-2026-06-22--unicode-文本规范化与-slug)
1617
- [本次更新 (2026-06-22) — JSON-Schema 兼容性检查](#本次更新-2026-06-22--json-schema-兼容性检查)
1718
- [本次更新 (2026-06-22) — 具类型的配置结构](#本次更新-2026-06-22--具类型的配置结构)
@@ -149,6 +150,12 @@
149150

150151
---
151152

153+
## 本次更新 (2026-06-22) — 时间序列变换
154+
155+
把计数器转成速率;降采样与重采样。完整参考:[`docs/source/Zh/doc/new_features/v98_features_doc.rst`](../docs/source/Zh/doc/new_features/v98_features_doc.rst)。
156+
157+
- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`**(`AC_ts_rate`、`AC_ts_downsample`):`observability` 计数器只存当前值(无处可把计数器转速率),`cost_telemetry` 只以天分桶。本功能在 `(timestamp, value)` 序列上加入 Prometheus 风格、具重置感知的 rate/increase/delta、tumbling-bucket 降采样(avg/sum/min/max/first/last/count)与网格重采样(last/linear/none)。不读 wall clock、确定。纯标准库。
158+
152159
## 本次更新 (2026-06-22) — Unicode 文本规范化与 Slug
153160

154161
在 fuzzy/search/OCR 匹配前规范化文本。完整参考:[`docs/source/Zh/doc/new_features/v97_features_doc.rst`](../docs/source/Zh/doc/new_features/v97_features_doc.rst)。

‎README/README_zh-TW.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212

1313
## 目錄
1414

15+
- [本次更新 (2026-06-22) — 時間序列轉換](#本次更新-2026-06-22--時間序列轉換)
1516
- [本次更新 (2026-06-22) — Unicode 文字正規化與 Slug](#本次更新-2026-06-22--unicode-文字正規化與-slug)
1617
- [本次更新 (2026-06-22) — JSON-Schema 相容性檢查](#本次更新-2026-06-22--json-schema-相容性檢查)
1718
- [本次更新 (2026-06-22) — 具型別的設定結構](#本次更新-2026-06-22--具型別的設定結構)
@@ -149,6 +150,12 @@
149150

150151
---
151152

153+
## 本次更新 (2026-06-22) — 時間序列轉換
154+
155+
把計數器轉成速率;降採樣與重採樣。完整參考:[`docs/source/Zh/doc/new_features/v98_features_doc.rst`](../docs/source/Zh/doc/new_features/v98_features_doc.rst)。
156+
157+
- **`ts_rate` / `ts_irate` / `ts_increase` / `ts_delta` / `ts_downsample` / `ts_resample`**(`AC_ts_rate`、`AC_ts_downsample`):`observability` 計數器只存當前值(無處可把計數器轉速率),`cost_telemetry` 只以天分桶。本功能在 `(timestamp, value)` 序列上加入 Prometheus 風格、具重置感知的 rate/increase/delta、tumbling-bucket 降採樣(avg/sum/min/max/first/last/count)與網格重採樣(last/linear/none)。不讀 wall clock、具決定性。純標準函式庫。
158+
152159
## 本次更新 (2026-06-22) — Unicode 文字正規化與 Slug
153160

154161
在 fuzzy/search/OCR 比對前正規化文字。完整參考:[`docs/source/Zh/doc/new_features/v97_features_doc.rst`](../docs/source/Zh/doc/new_features/v97_features_doc.rst)。

‎docs/source/Eng/doc/new_features/v98_features_doc.rst‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
Time-Series Transforms
2+
======================
3+
4+
``observability`` counters and gauges store only the *current* value — nothing
5+
turned a counter into a per-second rate — and ``cost_telemetry`` only buckets by
6+
a fixed day. This adds Prometheus-style ``rate`` / ``irate`` / ``increase`` /
7+
``delta`` (reset-aware) plus tumbling-bucket ``downsample`` and grid
8+
``resample`` over ``(timestamp, value)`` sequences.
9+
10+
Pure standard library (``bisect``); imports no ``PySide6``. No wall clock is
11+
read — windows use the series' own timestamps — so every function is fully
12+
deterministic in CI.
13+
14+
Headless API
15+
------------
16+
17+
.. code-block:: python
18+
19+
from je_auto_control import ts_rate, ts_increase, ts_downsample, ts_resample
20+
21+
series = [(0, 0), (10, 50), (20, 120)] # (timestamp_s, counter_value)
22+
ts_rate(series) # 6.0 (120 over 20s)
23+
ts_rate(series, window_s=10) # rate over the last 10s only
24+
ts_increase(series) # 120.0 (reset-aware)
25+
26+
ts_downsample([(0, 1), (3, 3), (5, 10)], 5, "avg") # [(0, 2.0), (5, 10.0)]
27+
ts_resample([(0, 0), (20, 20)], 10, fill="linear") # [(0,0),(10,10),(20,20)]
28+
29+
``ts_rate`` / ``ts_increase`` treat a value drop as a counter reset (Prometheus
30+
semantics); ``ts_irate`` is the instant rate from the last two samples;
31+
``ts_delta`` / ``ts_idelta`` are gauge first-to-last and last-two differences.
32+
``ts_downsample`` rolls the series into ``bucket_s`` tumbling buckets aggregated
33+
by ``avg`` / ``sum`` / ``min`` / ``max`` / ``first`` / ``last`` / ``count``.
34+
``ts_resample`` aligns to a fixed grid, filling with ``"last"`` (carry forward),
35+
``"linear"`` (interpolate), or ``None`` (gaps).
36+
37+
Executor commands
38+
-----------------
39+
40+
``AC_ts_rate`` returns ``{rate}`` for a ``series`` (optional ``window_s``);
41+
``AC_ts_downsample`` returns ``{buckets}`` for a ``series`` and ``bucket_s``
42+
(optional ``agg``). Both are exposed as MCP tools (``ac_ts_rate`` /
43+
``ac_ts_downsample``) and as Script Builder commands under **Data**.

‎docs/source/Eng/eng_index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ Comprehensive guides for all AutoControl features.
120120
doc/new_features/v95_features_doc
121121
doc/new_features/v96_features_doc
122122
doc/new_features/v97_features_doc
123+
doc/new_features/v98_features_doc
123124
doc/ocr_backends/ocr_backends_doc
124125
doc/observability/observability_doc
125126
doc/operations_layer/operations_layer_doc

‎docs/source/Zh/doc/new_features/v98_features_doc.rst‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
時間序列轉換
2+
==========
3+
4+
``observability`` 的計數器與量規只儲存*當前*值 —— 沒有任何東西能把計數器轉成每秒速率 —— 而
5+
``cost_telemetry`` 只以固定的「天」分桶。本功能在 ``(timestamp, value)`` 序列上加入 Prometheus 風格的
6+
``rate`` / ``irate`` / ``increase`` / ``delta``(具重置感知),以及 tumbling-bucket ``downsample`` 與
7+
網格 ``resample``。
8+
9+
純標準函式庫(``bisect``);不匯入 ``PySide6``。不讀取 wall clock —— 視窗使用序列自身的時間戳 —— 因此每個
10+
函式皆完全具決定性。
11+
12+
無頭 API
13+
--------
14+
15+
.. code-block:: python
16+
17+
from je_auto_control import ts_rate, ts_increase, ts_downsample, ts_resample
18+
19+
series = [(0, 0), (10, 50), (20, 120)] # (timestamp_s, counter_value)
20+
ts_rate(series) # 6.0(20 秒內 120)
21+
ts_rate(series, window_s=10) # 只看最後 10 秒的速率
22+
ts_increase(series) # 120.0(重置感知)
23+
24+
ts_downsample([(0, 1), (3, 3), (5, 10)], 5, "avg") # [(0, 2.0), (5, 10.0)]
25+
ts_resample([(0, 0), (20, 20)], 10, fill="linear") # [(0,0),(10,10),(20,20)]
26+
27+
``ts_rate`` / ``ts_increase`` 把值下降視為計數器重置(Prometheus 語意);``ts_irate`` 是最後兩個樣本的
28+
瞬時速率;``ts_delta`` / ``ts_idelta`` 是量規的首尾差與最後兩點差。``ts_downsample`` 把序列滾成 ``bucket_s``
29+
的 tumbling 桶,以 ``avg`` / ``sum`` / ``min`` / ``max`` / ``first`` / ``last`` / ``count`` 聚合。
30+
``ts_resample`` 對齊到固定網格,以 ``"last"``(前向填補)、``"linear"``(內插)或 ``None``(留缺)填值。
31+
32+
執行器命令
33+
----------
34+
35+
``AC_ts_rate`` 對 ``series``(可選 ``window_s``)回傳 ``{rate}``;``AC_ts_downsample`` 對 ``series`` 與
36+
``bucket_s``(可選 ``agg``)回傳 ``{buckets}``。兩者皆以 MCP 工具(``ac_ts_rate`` / ``ac_ts_downsample``)
37+
以及 Script Builder 中 **Data** 分類下的命令提供。

‎docs/source/Zh/zh_index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ AutoControl 所有功能的完整使用指南。
120120
doc/new_features/v95_features_doc
121121
doc/new_features/v96_features_doc
122122
doc/new_features/v97_features_doc
123+
doc/new_features/v98_features_doc
123124
doc/ocr_backends/ocr_backends_doc
124125
doc/observability/observability_doc
125126
doc/operations_layer/operations_layer_doc

‎je_auto_control/__init__.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -414,6 +414,11 @@
414414
)
415415
# Mergeable streaming latency digest + exact percentiles
416416
from je_auto_control.utils.percentiles import LatencyDigest, exact_percentiles
417+
# Time-series transforms: rate / irate / delta / downsample / resample
418+
from je_auto_control.utils.timeseries import (
419+
ts_delta, ts_downsample, ts_idelta, ts_increase, ts_irate, ts_rate,
420+
ts_resample,
421+
)
417422
# Bulkhead concurrency isolation + rate-limit header parsing
418423
from je_auto_control.utils.bulkhead import (
419424
Bulkhead, BulkheadFullError, next_delay, parse_ratelimit, parse_retry_after,
@@ -978,6 +983,8 @@ def start_autocontrol_gui(*args, **kwargs):
978983
"run_experiment",
979984
"BurnRule", "burn_alerts", "burn_rate", "default_burn_rules", "evaluate_slo",
980985
"LatencyDigest", "exact_percentiles",
986+
"ts_delta", "ts_downsample", "ts_idelta", "ts_increase", "ts_irate",
987+
"ts_rate", "ts_resample",
981988
"Bulkhead", "BulkheadFullError", "next_delay", "parse_ratelimit",
982989
"parse_retry_after",
983990
"Cassette", "CassetteMissError",

‎je_auto_control/gui/script_builder/command_schema.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1921,6 +1921,26 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
19211921
),
19221922
description="Classify JSON-Schema changes as backward/forward/full.",
19231923
))
1924+
specs.append(CommandSpec(
1925+
"AC_ts_rate", "Data", "Time-Series: Counter Rate",
1926+
fields=(
1927+
FieldSpec("series", FieldType.STRING,
1928+
placeholder="[[0, 0], [10, 50], [20, 120]]"),
1929+
FieldSpec("window_s", FieldType.FLOAT, optional=True),
1930+
),
1931+
description="Per-second counter rate (reset-aware) over a series.",
1932+
))
1933+
specs.append(CommandSpec(
1934+
"AC_ts_downsample", "Data", "Time-Series: Downsample",
1935+
fields=(
1936+
FieldSpec("series", FieldType.STRING,
1937+
placeholder="[[0, 1], [5, 3], [12, 9]]"),
1938+
FieldSpec("bucket_s", FieldType.FLOAT, placeholder="10"),
1939+
FieldSpec("agg", FieldType.STRING, optional=True,
1940+
placeholder="avg|sum|min|max|first|last|count"),
1941+
),
1942+
description="Roll a series into tumbling buckets by aggregate.",
1943+
))
19241944
specs.append(CommandSpec(
19251945
"AC_diff_rows", "Data", "Dataset Diff: Rows by Key",
19261946
fields=(

‎je_auto_control/utils/executor/action_executor.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3369,6 +3369,27 @@ def _percentiles(samples: Any, qs: Any = None) -> Dict[str, Any]:
33693369
return {"percentiles": {str(q): value for q, value in result.items()}}
33703370

33713371

3372+
def _ts_rate(series: Any, window_s: Any = None) -> Dict[str, Any]:
3373+
"""Adapter: per-second counter rate over a (ts, value) series."""
3374+
import json
3375+
from je_auto_control.utils.timeseries import ts_rate
3376+
if isinstance(series, str):
3377+
series = json.loads(series)
3378+
window = float(window_s) if window_s is not None else None
3379+
return {"rate": ts_rate(series, window_s=window)}
3380+
3381+
3382+
def _ts_downsample(series: Any, bucket_s: Any,
3383+
agg: str = "avg") -> Dict[str, Any]:
3384+
"""Adapter: downsample a (ts, value) series into tumbling buckets."""
3385+
import json
3386+
from je_auto_control.utils.timeseries import ts_downsample
3387+
if isinstance(series, str):
3388+
series = json.loads(series)
3389+
buckets = ts_downsample(series, float(bucket_s), agg)
3390+
return {"buckets": [list(point) for point in buckets]}
3391+
3392+
33723393
def _evaluate_slo(records: Any, target: float,
33733394
window_s: Optional[float] = None) -> Dict[str, Any]:
33743395
"""Adapter: SLI + error budget for outcome records (list or JSON string)."""
@@ -4464,6 +4485,8 @@ def __init__(self):
44644485
"AC_resolve_config": _resolve_config,
44654486
"AC_explain_config": _explain_config,
44664487
"AC_check_compatibility": _check_compatibility,
4488+
"AC_ts_rate": _ts_rate,
4489+
"AC_ts_downsample": _ts_downsample,
44674490
"AC_detect_drift": _detect_drift,
44684491
"AC_categorical_drift": _categorical_drift,
44694492
"AC_diff_rows": _diff_rows,

0 commit comments

Comments
 (0)