feat(orchestrator): show token and commit rates in the header
rates.py folds the token, commit and score journals into per-hour and per-day rates for the header chips and the rates page; /api/rates drops from about 1950 ms to 380-830 ms on a year of data. Tests run once on unify/skills-20261114 after the merge (operator order 2026-11-14). 🎯 Fix: 75 % 💥 Break: 10 % 💸 Tokens: – ⏱️ Velocity: – 🛡️ Security: 95 % 🏆 Score: 64 🚦 Gate: ⛔ fix 75 % < 90, break 10 % > 5; security 95 % < 100: ask the operator before the commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a2a03995cb
commit
652bc817c1
28 files changed
+1449
-102
No files matched your search
@@ -0,0 +1,113 @@
|
||||
# 012 - Header Rates
|
||||
|
||||
## User Story
|
||||
|
||||
As the operator, I want a chip in the hub's header that shows how much work
|
||||
gets done, how many tokens it costs and how loaded the host is per minute,
|
||||
hour, day, week, month, year or in total, so that I see the pace of the agents
|
||||
at a glance on every screen.
|
||||
|
||||
## Decisions (operator-confirmed 2026-11-14; do not re-litigate)
|
||||
|
||||
1. **Rates per period**: the operator picks minute, hour, day, week, month,
|
||||
year or total. Per period the chip shows the work items as three separate
|
||||
kinds (register tasks done, commits, key results met), the tokens, and
|
||||
the average CPU %, RAM %, VRAM (used and %), swap % and disk usage over
|
||||
that period. Total means since records begin: totals for counts,
|
||||
all-time averages for gauges.
|
||||
2. **Exact windows on the server**: `/api/rates` returns one small JSON for
|
||||
all periods, each window ending now. Sources: the sampler's
|
||||
`samples.jsonl` and `tokens.jsonl`, the register's `rasci.json` (a task's
|
||||
`end` once done) and the score journal `scores.jsonl` (a row with a sha is
|
||||
a commit, counted once per remote and sha); the hub's existing readers
|
||||
read them, no second parser.
|
||||
3. **Daily buckets kept forever**: the sampler's retention folds rows older
|
||||
than 90 days into one row per day, kept forever, with `mean_row` and the
|
||||
bucket folding; a folded row carries `span`, the seconds its rows sampled,
|
||||
and `width`, its bucket's seconds; the gauges weight every row by its span
|
||||
(a raw row 60 s) and a row whose bucket straddles a window start by the
|
||||
share inside. Token rows of a day that ended more than 90 days ago fold
|
||||
into one row per day, agent and model with `width`, kept forever; a window
|
||||
reaching back before the first token row is an estimate (≈, source total)
|
||||
scaled from the token rows there are.
|
||||
4. **KR met timestamp**: `rasci.py kr <task> met <n>` records `met_at` with
|
||||
`stamp()` when the key result turns met, `open <n>` drops it. A key result
|
||||
met without `met_at` stays so when met again, never counts in a window,
|
||||
but counts in total. The commit table after a commit (`score.py
|
||||
--commit-table <repo> --commit <sha>`) appends a journal row with the full
|
||||
sha and the commit time unless one for that remote and sha exists, so the
|
||||
commits count.
|
||||
5. **Extrapolation from the next longer window**: a period without an event
|
||||
of a kind takes the shortest longer window (hour, day, week, ..., total)
|
||||
that holds some and scales it to the period; the chip shows it with ≈ and
|
||||
names the source window in a tooltip, the API with
|
||||
`approx: true, source: "<period>"`. Token rows are hourly, so the minute
|
||||
always takes a longer window. Gauges never extrapolate; a gauge without a
|
||||
sample shows n/a.
|
||||
6. **Chip and popover**: the chip shows the period picker and 2 to 3 chosen
|
||||
metrics; a click opens a table of every metric and every period. Period
|
||||
and chip metrics persist in the hub settings (`rates`), which need no
|
||||
passkey. The chip fetches `/api/rates` on the page's refresh tick at most
|
||||
every 60 s.
|
||||
|
||||
## Approach
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph P["🧭 0 · Preparation"]
|
||||
P1["📋 Write requirement"] --> P2{"✅ Operator approves?"}
|
||||
P2 -- "✏️ corrections" --> P1
|
||||
end
|
||||
P2 -- "🤖 yes" --> A1
|
||||
subgraph A["🗄️ A · Data"]
|
||||
A1["🧩 Daily folding, met_at, /api/rates, settings"] --> A2{"🧪 Unit and integration green?"}
|
||||
A2 -- "🔴 no: fix" --> A1
|
||||
end
|
||||
A2 -- "🟢 yes" --> B1
|
||||
subgraph B["📊 B · Chip"]
|
||||
B1["🏷️ Header chip, popover table"] --> B2{"🧪 E2E green?"}
|
||||
B2 -- "🔴 no: fix" --> B1
|
||||
B2 -- "🟢 yes" --> BF{"💬 Operator feedback on screenshots?"}
|
||||
BF -- "✏️ changes" --> B1
|
||||
end
|
||||
BF -- "👍 fits" --> D1["🏁 Review, commit with operator OK, hand-over"]
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Data
|
||||
|
||||
- [ ] Each period counts the events inside its window; total counts every
|
||||
event, also those without a time.
|
||||
- [ ] A token row counts in a window by the share of its hour inside it.
|
||||
- [ ] Gauges average every window across raw rows, 15-minute means and daily
|
||||
means, each weighted by its span.
|
||||
- [ ] An empty minute takes the hour, an empty hour the day, marked approx
|
||||
with the source; a gauge without a sample is null.
|
||||
- [ ] Retention folds days older than 90 days into one row each, idempotent,
|
||||
and keeps them forever.
|
||||
- [ ] `met_at` is set when a key result turns met, kept on a second met,
|
||||
never added to one met before it existed, and dropped on open.
|
||||
|
||||
### Chip
|
||||
|
||||
- [ ] The chip sits right of the Live chip, shows the chosen period and
|
||||
metrics and never widens the header at 1920, 1366 or 390.
|
||||
- [ ] A period change and a metric swap persist over a reload.
|
||||
- [ ] The popover opens on click and closes on Escape, a second click and a
|
||||
click outside.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Unit: window sums and averages per period and row tier,
|
||||
extrapolation, daily folding, `met_at`, the settings round trip.
|
||||
- [ ] Integration: the hub serves `/api/rates` from fixture files.
|
||||
- [ ] E2E: Chromium and Firefox at 1920, 1366 and 390: chip, persisted
|
||||
period, popover open and close; screenshots for the checkpoint.
|
||||
|
||||
## Known Limits
|
||||
|
||||
- Tokens before the first token row are unknown; the windows reaching back
|
||||
before it show a scaled estimate.
|
||||
- A cold `/api/rates` call decodes every sample row; on a year of samples the
|
||||
JSON decoding alone takes about 180 ms.
|
||||
@@ -12,3 +12,4 @@ skill defines the format.
|
||||
- [006 - Score Metric Overlays](006-score-metric-overlays.md)
|
||||
- [007 - Resources View](007-resources-view.md)
|
||||
- [008 - Home Dashboard](008-home-dashboard.md)
|
||||
- [012 - Header Rates](012-header-rates.md)
|
||||
@@ -13,7 +13,9 @@
|
||||
reading for (no driver, a driver mismatch) keeps its name with util
|
||||
and vram None. In the first minute of a 15-minute bucket, rows older
|
||||
than 7 days fold into 15-minute means and means older than 90 days
|
||||
are dropped.
|
||||
into one mean per day, kept forever, each row weighted by the seconds
|
||||
it sampled; a folded row carries span, the seconds its rows sampled
|
||||
(60 per raw row), and width, the seconds of its bucket.
|
||||
tokens: add the input, output, cache-read and cache-write tokens of the
|
||||
usage fields in the Claude Code transcripts under --transcripts since
|
||||
each file's byte offset to <state>/tokens.jsonl as {t, agent, model,
|
||||
@@ -25,7 +27,8 @@
|
||||
seen: {hour: [key digest]}}, seen keeping the hours this run read and
|
||||
the last two, and is replaced only when it changes. In the first
|
||||
minute of an hour, rows of the same hour, agent and model merge and
|
||||
rows older than 90 days are dropped.
|
||||
the rows of a day that ended more than 90 days ago fold into one row
|
||||
per day, agent and model with width: 86400, kept forever.
|
||||
--state: the directory of these files, the unit passes
|
||||
~/.local/state/agents-council/resources.
|
||||
--council: the council directory with queue.json and the claims.
|
||||
@@ -61,8 +64,9 @@ import queue_live
|
||||
INTERVAL = 60
|
||||
BUCKET = 900
|
||||
HOUR = 3600
|
||||
FINE = 7 * 86400
|
||||
COARSE = 90 * 86400
|
||||
DAY = 86400
|
||||
FINE = 7 * DAY
|
||||
COARSE = 90 * DAY
|
||||
COMMAND_SECONDS = 10
|
||||
KEY_BYTES = 9
|
||||
KIB = 1 << 10
|
||||
@@ -278,77 +282,122 @@ def sample(proc: Path, sys_root: Path, council: Path, run: Callable[[list[str]],
|
||||
}
|
||||
|
||||
|
||||
def average(values: list[object], digits: int) -> float | None:
|
||||
found = [value for value in values if isinstance(value, int | float) and not isinstance(value, bool)]
|
||||
return round(sum(found) / len(found), digits) if found else None
|
||||
def is_number(value: object) -> bool:
|
||||
return isinstance(value, int | float) and not isinstance(value, bool)
|
||||
|
||||
|
||||
def whole(values: list[object]) -> int | None:
|
||||
found = average(values, 0)
|
||||
def weight(row: dict) -> int | float:
|
||||
"""Returns:
|
||||
The seconds row sampled: its span, INTERVAL for a raw row.
|
||||
"""
|
||||
span = row.get("span")
|
||||
return span if is_number(span) and span > 0 else INTERVAL
|
||||
|
||||
|
||||
def average(pairs: list[tuple[int | float, object]], digits: int) -> float | None:
|
||||
"""Args:
|
||||
pairs: (weight, value) per reading; a value that is no number is
|
||||
left out.
|
||||
"""
|
||||
found = [(part, value) for part, value in pairs if is_number(value)]
|
||||
total = sum(part for part, _ in found)
|
||||
return round(sum(part * value for part, value in found) / total, digits) if total else None
|
||||
|
||||
|
||||
def whole(pairs: list[tuple[int | float, object]]) -> int | None:
|
||||
found = average(pairs, 0)
|
||||
return None if found is None else int(found)
|
||||
|
||||
|
||||
def listed(rows: list[dict], key: str) -> list[dict]:
|
||||
return [item for row in rows if isinstance(row.get(key), list) for item in row[key] if isinstance(item, dict)]
|
||||
def listed(rows: list[dict], key: str) -> list[tuple[int | float, dict]]:
|
||||
return [(weight(row), item) for row in rows if isinstance(row.get(key), list) for item in row[key] if isinstance(item, dict)]
|
||||
|
||||
|
||||
def grouped(items: list[dict], key: str) -> dict[object, list[dict]]:
|
||||
found: dict[object, list[dict]] = {}
|
||||
for item in items:
|
||||
found.setdefault(item.get(key), []).append(item)
|
||||
def grouped(items: list[tuple[int | float, dict]], key: str) -> dict[object, list[tuple[int | float, dict]]]:
|
||||
found: dict[object, list[tuple[int | float, dict]]] = {}
|
||||
for part, item in items:
|
||||
found.setdefault(item.get(key), []).append((part, item))
|
||||
return found
|
||||
|
||||
|
||||
def gpu_mean(name: object, readings: list[dict]) -> dict:
|
||||
awake = [reading for reading in readings if reading.get("asleep") is not True]
|
||||
def means(readings: list[tuple[int | float, dict]], key: str) -> list[tuple[int | float, object]]:
|
||||
return [(part, reading.get(key)) for part, reading in readings]
|
||||
|
||||
|
||||
def gpu_mean(name: object, readings: list[tuple[int | float, dict]]) -> dict:
|
||||
awake = [(part, reading) for part, reading in readings if reading.get("asleep") is not True]
|
||||
if not awake:
|
||||
return {"name": name, "asleep": True}
|
||||
return {"name": name, "util": average([reading.get("util") for reading in awake], 1), "vram_used": whole([reading.get("vram_used") for reading in awake]), "vram_total": whole([reading.get("vram_total") for reading in awake])}
|
||||
return {"name": name, "util": average(means(awake, "util"), 1), "vram_used": whole(means(awake, "vram_used")), "vram_total": whole(means(awake, "vram_total"))}
|
||||
|
||||
|
||||
def disk_means(rows: list[dict]) -> list[dict]:
|
||||
return [{"mount": mount, "used": whole([disk.get("used") for disk in disks]), "total": whole([disk.get("total") for disk in disks])} for mount, disks in grouped(listed(rows, "disks"), "mount").items()]
|
||||
return [{"mount": mount, "used": whole(means(disks, "used")), "total": whole(means(disks, "total"))} for mount, disks in grouped(listed(rows, "disks"), "mount").items()]
|
||||
|
||||
|
||||
def queue_mean(rows: list[dict]) -> dict:
|
||||
queues = [row["queue"] for row in rows if isinstance(row.get("queue"), dict)]
|
||||
return {key: average([entry.get(key) for entry in queues], 2) for key in ("waiting", "active")}
|
||||
queues = [(weight(row), row["queue"]) for row in rows if isinstance(row.get("queue"), dict)]
|
||||
return {key: average(means(queues, key), 2) for key in ("waiting", "active")}
|
||||
|
||||
|
||||
def mean_row(start: int, rows: list[dict]) -> dict:
|
||||
def mean_row(start: int, width: int, rows: list[dict]) -> dict:
|
||||
"""Returns:
|
||||
The 15-minute mean of rows at start: per mount and per GPU the mean
|
||||
of the rows that list it, a GPU asleep in every row stays asleep;
|
||||
a single row already at start comes back unchanged.
|
||||
The mean of rows in the bucket of width seconds at start, each row
|
||||
weighted by weight(): per mount and per GPU the mean of the rows that
|
||||
list it, a GPU asleep in every row stays asleep; span the seconds the
|
||||
rows sampled together, width the bucket's.
|
||||
"""
|
||||
host = {key: average([(weight(row), row.get(key)) for row in rows], digits) for key, digits in HOST_DIGITS}
|
||||
gpu_rows = [gpu_mean(name, readings) for name, readings in grouped(listed(rows, "gpus"), "name").items()]
|
||||
return {"t": start, "disks": disk_means(rows), **host, "gpus": gpu_rows, "queue": queue_mean(rows), "span": sum(weight(row) for row in rows), "width": width}
|
||||
|
||||
|
||||
def folded(start: int, width: int, rows: list[dict]) -> dict:
|
||||
"""Returns:
|
||||
The one row the bucket of width seconds at start keeps: a lone row
|
||||
at start already folded to width or wider unchanged; a lone row at
|
||||
start without span, a 15-minute mean written before folded rows
|
||||
carried span, with span and width the whole bucket; else mean_row().
|
||||
"""
|
||||
if len(rows) == 1 and rows[0]["t"] == start:
|
||||
return rows[0]
|
||||
host = {key: average([row.get(key) for row in rows], digits) for key, digits in HOST_DIGITS}
|
||||
gpu_rows = [gpu_mean(name, readings) for name, readings in grouped(listed(rows, "gpus"), "name").items()]
|
||||
return {"t": start, "disks": disk_means(rows), **host, "gpus": gpu_rows, "queue": queue_mean(rows)}
|
||||
kept = rows[0].get("width")
|
||||
if is_number(kept) and kept >= width:
|
||||
return rows[0]
|
||||
if "span" not in rows[0]:
|
||||
return {**rows[0], "span": width, "width": width}
|
||||
return mean_row(start, width, rows)
|
||||
|
||||
|
||||
def timed(row: object) -> bool:
|
||||
return isinstance(row, dict) and isinstance(row.get("t"), int | float) and not isinstance(row.get("t"), bool)
|
||||
return isinstance(row, dict) and is_number(row.get("t"))
|
||||
|
||||
|
||||
def fold(rows: list[dict], width: int, line: float) -> list[dict]:
|
||||
"""Returns:
|
||||
rows with every bucket of width seconds that ended by line folded()
|
||||
into one row at its start, the other rows following unchanged and in
|
||||
order.
|
||||
"""
|
||||
buckets: dict[int, list[dict]] = {}
|
||||
rest = []
|
||||
for row in rows:
|
||||
start = int(row["t"] // width * width)
|
||||
if start + width <= line:
|
||||
buckets.setdefault(start, []).append(row)
|
||||
else:
|
||||
rest.append(row)
|
||||
return [folded(start, width, buckets[start]) for start in sorted(buckets)] + rest
|
||||
|
||||
|
||||
def retain(rows: list[dict], now: float) -> list[dict]:
|
||||
"""Returns:
|
||||
rows without those older than 90 days, every 15-minute bucket that
|
||||
ended more than 7 days ago folded into one mean at its start, the
|
||||
newer rows unchanged and in order.
|
||||
rows with every 15-minute bucket that ended more than 7 days ago
|
||||
folded into one mean at its start, every day (UTC) that ended more
|
||||
than 90 days ago folded into one mean of those, kept forever, and the
|
||||
newer rows unchanged and in order; a folded row carries span, the
|
||||
seconds its rows sampled, and width, its bucket's seconds.
|
||||
"""
|
||||
buckets: dict[int, list[dict]] = {}
|
||||
recent = []
|
||||
for row in rows:
|
||||
if not timed(row) or row["t"] < now - COARSE:
|
||||
continue
|
||||
start = int(row["t"] // BUCKET * BUCKET)
|
||||
if start + BUCKET <= now - FINE:
|
||||
buckets.setdefault(start, []).append(row)
|
||||
else:
|
||||
recent.append(row)
|
||||
return [mean_row(start, buckets[start]) for start in sorted(buckets)] + recent
|
||||
return fold(fold([row for row in rows if timed(row)], BUCKET, now - FINE), DAY, now - COARSE)
|
||||
|
||||
|
||||
def read_rows(path: Path) -> list[dict]:
|
||||
@@ -498,7 +547,25 @@ def scan(path: Path, start: int, budget: int, seen: dict[int, set[str]]) -> tupl
|
||||
|
||||
|
||||
def token_rows(sums: dict[tuple, list[int]]) -> list[dict]:
|
||||
return [{"t": t, "agent": agent, "model": model, **dict(zip(NAMES, counts, strict=True))} for (t, agent, model), counts in sorted(sums.items())]
|
||||
"""Args:
|
||||
sums: per (t, agent, model, width) the counts in COUNTS order.
|
||||
|
||||
Returns:
|
||||
One token row per key in key order; a row other than hourly names
|
||||
its width.
|
||||
"""
|
||||
return [{"t": t, "agent": agent, "model": model, **dict(zip(NAMES, counts, strict=True)), **({} if width == HOUR else {"width": width})} for (t, agent, model, width), counts in sorted(sums.items())]
|
||||
|
||||
|
||||
def token_key(row: dict, line: float) -> tuple:
|
||||
"""Returns:
|
||||
(t, agent, model, width) of the row a token row merges into: its day
|
||||
(UTC) when the row is daily or the day ended by line, else its hour.
|
||||
"""
|
||||
day = int(row["t"] // DAY * DAY)
|
||||
if row.get("width") == DAY or day + DAY <= line:
|
||||
return day, row["agent"], row["model"], DAY
|
||||
return row["t"], row["agent"], row["model"], HOUR
|
||||
|
||||
|
||||
def add(sums: dict[tuple, list[int]], key: tuple, counts: tuple[int, ...]) -> None:
|
||||
@@ -508,10 +575,14 @@ def add(sums: dict[tuple, list[int]], key: tuple, counts: tuple[int, ...]) -> No
|
||||
|
||||
|
||||
def compact(rows: list[dict], now: float) -> list[dict]:
|
||||
"""Returns:
|
||||
The token rows merged per token_key(), every day that ended more than
|
||||
90 days ago one row per agent and model, kept forever.
|
||||
"""
|
||||
sums: dict[tuple, list[int]] = {}
|
||||
for row in rows:
|
||||
if timed(row) and row["t"] >= now - COARSE and isinstance(row.get("agent"), str) and isinstance(row.get("model"), str):
|
||||
add(sums, (row["t"], row["agent"], row["model"]), tuple(count(row.get(name)) for name in NAMES))
|
||||
if timed(row) and isinstance(row.get("agent"), str) and isinstance(row.get("model"), str):
|
||||
add(sums, token_key(row, now - COARSE), tuple(count(row.get(name)) for name in NAMES))
|
||||
return token_rows(sums)
|
||||
|
||||
|
||||
@@ -568,7 +639,7 @@ def count_tokens(transcripts: Path, council: Path, state: Path, now: float, budg
|
||||
seen.setdefault(hour, set()).update(read)
|
||||
touched.add(hour)
|
||||
for ts, session, model, counts in found:
|
||||
add(sums, (int(ts // HOUR * HOUR), names.get(session) or session[:8], model), counts)
|
||||
add(sums, (int(ts // HOUR * HOUR), names.get(session) or session[:8], model, HOUR), counts)
|
||||
rows = token_rows(sums)
|
||||
append_rows(state / TOKENS, rows)
|
||||
kept = {"files": files, "seen": kept_seen(seen, touched, now)}
|
||||
|
||||
@@ -10,6 +10,7 @@ from urllib.parse import urlsplit
|
||||
from .diffs import DIFF_BODY_LIMIT
|
||||
from .engines import STT_LIMIT
|
||||
from .exchange import PLAIN_CSP, Exchange
|
||||
from .rates import rated
|
||||
from .resources import RESOURCES, sampled
|
||||
from .scores import scored
|
||||
|
||||
@@ -131,6 +132,8 @@ class Handler(Exchange):
|
||||
self.answer(lambda: sampled(hub.kept / RESOURCES, hub.span(self.query(), time.time())), HTTPStatus.OK)
|
||||
elif path == "/api/scores":
|
||||
self.answer(lambda: scored(hub.council, hub.scripts.skills, hub.span(self.query(), time.time())), HTTPStatus.OK)
|
||||
elif path == "/api/rates":
|
||||
self.answer(lambda: rated(hub.council, hub.kept, time.time()), HTTPStatus.OK)
|
||||
elif path == VIEW_ROUTE:
|
||||
self.answer(hub.view_store.get, HTTPStatus.OK)
|
||||
elif path == "/api/subagents":
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
|
||||
from .records import epoch, iso_epoch
|
||||
from .register import OPERATOR, REGISTER, read_register, status_of
|
||||
from .resources import HOUR, RESOURCES, SAMPLES, TOKENS, rows_between
|
||||
from .scores import journal
|
||||
|
||||
DAY = 86400
|
||||
PERIODS = (("minute", 60), ("hour", HOUR), ("day", DAY), ("week", 7 * DAY), ("month", 30 * DAY), ("year", 365 * DAY), ("total", None))
|
||||
PERIOD_NAMES = tuple(name for name, _ in PERIODS)
|
||||
COUNTS = ("tasks", "commits", "krs", "tokens")
|
||||
GAUGES = ("cpu", "ram", "vram", "swap", "disk")
|
||||
METRICS = (*COUNTS, *GAUGES)
|
||||
HOST = ("cpu", "ram", "swap")
|
||||
TOKEN_FIELDS = ("input", "output", "cache_read", "cache_write")
|
||||
FINEST = {"tokens": HOUR}
|
||||
INTERVAL = 60
|
||||
DONE = "done"
|
||||
DIGITS = 4
|
||||
TOTAL = "total"
|
||||
|
||||
NUMBERS = (int, float)
|
||||
LIMIT = 1e18
|
||||
|
||||
Piece = tuple[float | None, float | None, float]
|
||||
Placed = tuple[float, float, dict]
|
||||
|
||||
|
||||
def rounded(value: float | None, digits: int) -> float | int | None:
|
||||
if value is None:
|
||||
return None
|
||||
return round(value) if digits == 0 else round(value, digits)
|
||||
|
||||
|
||||
def points(times: Iterable[float | None]) -> list[Piece]:
|
||||
return [(t, t, 1.0) for t in times]
|
||||
|
||||
|
||||
def task_times(tasks: dict[str, dict]) -> list[float | None]:
|
||||
"""Returns:
|
||||
The end of every done register task but the operator's commands,
|
||||
None for one without a readable end.
|
||||
"""
|
||||
return [iso_epoch(row.get("end")) for task, row in tasks.items() if not task.startswith(OPERATOR) and status_of(row) == DONE]
|
||||
|
||||
|
||||
def kr_times(tasks: dict[str, dict]) -> list[float | None]:
|
||||
"""Returns:
|
||||
The met_at of every met key result, None for one met before
|
||||
rasci.py recorded that time.
|
||||
"""
|
||||
results = [kr for row in tasks.values() if isinstance(row.get("key_results"), list) for kr in row["key_results"]]
|
||||
return [iso_epoch(kr.get("met_at")) for kr in results if isinstance(kr, dict) and kr.get("met") is True]
|
||||
|
||||
|
||||
def commit_times(rows: list[dict]) -> list[float | None]:
|
||||
"""Returns:
|
||||
The time of every commit the score journal holds a row with a sha
|
||||
for, once per remote and sha.
|
||||
"""
|
||||
found: dict[tuple[str | None, str], float | None] = {}
|
||||
for row in rows:
|
||||
sha, remote = row.get("sha"), row.get("remote")
|
||||
if isinstance(sha, str) and sha:
|
||||
found.setdefault((remote if isinstance(remote, str) else None, sha), epoch(row.get("time")))
|
||||
return list(found.values())
|
||||
|
||||
|
||||
def token_pieces(rows: list[dict], now: float) -> list[Piece]:
|
||||
"""Returns:
|
||||
Per token row its hour, or its day for a row with width, cut at now,
|
||||
and its tokens of every kind.
|
||||
"""
|
||||
found = []
|
||||
for row in rows:
|
||||
t = epoch(row.get("t"))
|
||||
width = row["width"] if sane(row.get("width")) and row["width"] > 0 else HOUR
|
||||
amount = sum(value for name in TOKEN_FIELDS if isinstance(value := row.get(name), int) and not isinstance(value, bool) and value > 0)
|
||||
if t is not None and amount:
|
||||
found.append((t, max(t, min(t + width, now)), float(amount)))
|
||||
return found
|
||||
|
||||
|
||||
def share(piece: Piece, begin: float, now: float) -> float:
|
||||
"""Returns:
|
||||
The part of the piece's amount inside [begin, now]: an instant
|
||||
counts whole or not at all, a span by the share of it inside.
|
||||
"""
|
||||
start, end, amount = piece
|
||||
if start is None or end is None:
|
||||
return 0.0
|
||||
if end <= start:
|
||||
return amount if begin <= start <= now else 0.0
|
||||
inside = min(end, now) - max(start, begin)
|
||||
return amount * inside / (end - start) if inside > 0 else 0.0
|
||||
|
||||
|
||||
def sums(pieces: list[Piece], now: float, finest: float) -> dict[str, float | None]:
|
||||
"""Args:
|
||||
pieces: (start, end, amount) per event; start None for one without
|
||||
a time, end equal to start for an instant.
|
||||
now: the end of every window.
|
||||
finest: the shortest window the pieces resolve; a shorter one holds
|
||||
no event of its own.
|
||||
|
||||
Returns:
|
||||
Per period the amount inside its window, None below finest; total
|
||||
holds every piece.
|
||||
"""
|
||||
found: dict[str, float | None] = {}
|
||||
for name, seconds in PERIODS:
|
||||
if seconds is None:
|
||||
found[name] = sum(piece[2] for piece in pieces)
|
||||
else:
|
||||
found[name] = None if seconds < finest else sum(share(piece, now - seconds, now) for piece in pieces)
|
||||
return found
|
||||
|
||||
|
||||
def lengths(now: float, since: float | None) -> dict[str, float | None]:
|
||||
total = now - since if since is not None and now > since else None
|
||||
return {name: total if seconds is None else float(seconds) for name, seconds in PERIODS}
|
||||
|
||||
|
||||
def held(now: float, since: float | None, begins: float | None) -> dict[str, float | None]:
|
||||
"""Args:
|
||||
now: the end of every window.
|
||||
since: the first time any record holds.
|
||||
begins: the first time the records of one kind hold, None when they
|
||||
reach back as far as any record.
|
||||
|
||||
Returns:
|
||||
Per period the seconds now - begins where other records reach
|
||||
further back into its window than this kind's, else None.
|
||||
"""
|
||||
if since is None or begins is None:
|
||||
return dict.fromkeys(PERIOD_NAMES)
|
||||
return {name: now - begins if begins > (since if seconds is None else max(now - seconds, since)) else None for name, seconds in PERIODS}
|
||||
|
||||
|
||||
def estimate(own: dict[str, float | None], length: dict[str, float | None], partial: dict[str, float | None], index: int) -> tuple[float, str | None]:
|
||||
"""Returns:
|
||||
(value, source) of the period at index in PERIODS: its own amount
|
||||
when partial holds no seconds for it and it holds any or is total;
|
||||
its own amount scaled from the partial seconds to the part of its
|
||||
window any record covers when it holds any, source total; else the
|
||||
shortest longer window that holds any, scaled to the period's length
|
||||
(at most its whole amount), source that window, or total when
|
||||
partial holds seconds for that window and those scale it; 0 without
|
||||
any.
|
||||
"""
|
||||
name, seconds = PERIODS[index]
|
||||
value = own[name]
|
||||
if partial[name] is None and (value or seconds is None):
|
||||
return value or 0.0, None
|
||||
if value and partial[name]:
|
||||
return value * min(length[name], length[TOTAL]) / partial[name], TOTAL
|
||||
for other, _ in PERIODS[index + 1:]:
|
||||
if own[other] and length[other]:
|
||||
covered = partial[other] or length[other]
|
||||
return own[other] * min(1.0, (seconds or covered) / covered), TOTAL if partial[other] else other
|
||||
return 0.0, None
|
||||
|
||||
|
||||
def extrapolated(own: dict[str, float | None], length: dict[str, float | None], partial: dict[str, float | None]) -> dict[str, dict]:
|
||||
"""Args:
|
||||
own: sums() of one kind.
|
||||
length: lengths() of the windows.
|
||||
partial: held() of the kind.
|
||||
|
||||
Returns:
|
||||
Per period {value, approx, source} of estimate(); approx whenever it
|
||||
names a source.
|
||||
"""
|
||||
found = {}
|
||||
for index, name in enumerate(PERIOD_NAMES):
|
||||
value, source = estimate(own, length, partial, index)
|
||||
found[name] = {"value": precise(value), "approx": source is not None, "source": source}
|
||||
return found
|
||||
|
||||
|
||||
def precise(value: float) -> float:
|
||||
return round(value, DIGITS - 1) if value >= 1 else float(f"{value:.{DIGITS}g}")
|
||||
|
||||
|
||||
def sane(value: object) -> bool:
|
||||
return type(value) in NUMBERS and -LIMIT < value < LIMIT
|
||||
|
||||
|
||||
def placed(row: dict) -> Placed | None:
|
||||
"""Returns:
|
||||
(t, width, row) of a sample row: width the seconds of the bucket a
|
||||
folded row stands for (its span when it names no width, 0 for a raw
|
||||
row, an instant); None without a time.
|
||||
"""
|
||||
t = epoch(row.get("t"))
|
||||
if t is None:
|
||||
return None
|
||||
width, span = row.get("width"), row.get("span")
|
||||
return t, float(width if sane(width) and width > 0 else span if sane(span) and span > 0 else 0), row
|
||||
|
||||
|
||||
def blank() -> dict:
|
||||
return {**{key: [0.0, 0.0] for key in HOST}, "vram": [0.0, 0.0, 0.0], "disks": {}}
|
||||
|
||||
|
||||
def copied(tally: dict) -> dict:
|
||||
return {**{key: list(tally[key]) for key in (*HOST, "vram")}, "disks": {mount: list(cell) for mount, cell in tally["disks"].items()}}
|
||||
|
||||
|
||||
def add_vram(cell: list[float], gpus: object, weight: float) -> None:
|
||||
used = total = 0.0
|
||||
for gpu in gpus if type(gpus) is list else ():
|
||||
if type(gpu) is dict and gpu.get("asleep") is not True:
|
||||
part, whole = gpu.get("vram_used"), gpu.get("vram_total")
|
||||
if sane(part) and sane(whole) and whole > 0:
|
||||
used, total = used + part, total + whole
|
||||
if total:
|
||||
cell[0] += weight
|
||||
cell[1] += weight * 100 * used / total
|
||||
cell[2] += weight * used
|
||||
|
||||
|
||||
def add_disks(cells: dict[str, list[float]], disks: object, weight: float) -> None:
|
||||
for disk in disks if type(disks) is list else ():
|
||||
if type(disk) is dict:
|
||||
mount, used, total = disk.get("mount"), disk.get("used"), disk.get("total")
|
||||
if type(mount) is str and sane(used) and sane(total) and total > 0:
|
||||
cell = cells.get(mount) or cells.setdefault(mount, [0.0, 0.0, 0.0, 0.0])
|
||||
cell[0] += weight
|
||||
cell[1] += weight * 100 * used / total
|
||||
cell[2] += weight * used
|
||||
cell[3] += weight * total
|
||||
|
||||
|
||||
def add(tally: dict, row: dict, part: float) -> None:
|
||||
"""Args:
|
||||
tally: blank() summed so far: per host key [weight, weighted value],
|
||||
vram [weight, weighted percent, weighted bytes] of every GPU
|
||||
awake with a reading together, per mount [weight, weighted
|
||||
percent, weighted used, weighted total].
|
||||
row: one sample row, weighted by the seconds it sampled: its span,
|
||||
INTERVAL for a raw row.
|
||||
part: the share of the row inside the window.
|
||||
"""
|
||||
span = row.get("span")
|
||||
weight = (span if sane(span) and span > 0 else INTERVAL) * part
|
||||
for key in HOST:
|
||||
value = row.get(key)
|
||||
if sane(value):
|
||||
cell = tally[key]
|
||||
cell[0] += weight
|
||||
cell[1] += weight * value
|
||||
add_vram(tally["vram"], row.get("gpus"), weight)
|
||||
add_disks(tally["disks"], row.get("disks"), weight)
|
||||
|
||||
|
||||
def tallies(items: list[Placed], begins: list[float]) -> list[dict]:
|
||||
"""Args:
|
||||
items: placed() of every sample row, newest first.
|
||||
begins: the start of every window, latest first; every window ends
|
||||
at the newest row or later.
|
||||
|
||||
Returns:
|
||||
Per window the add() of every row inside it, a row whose bucket
|
||||
straddles its start by the share inside, in one pass over items.
|
||||
"""
|
||||
widest = max((item[1] for item in items), default=0.0)
|
||||
running, index, found = blank(), 0, []
|
||||
for begin in begins:
|
||||
while index < len(items) and items[index][0] >= begin:
|
||||
add(running, items[index][2], 1.0)
|
||||
index += 1
|
||||
tally = copied(running)
|
||||
cursor = index
|
||||
while cursor < len(items) and items[cursor][0] > begin - widest:
|
||||
t, width, row = items[cursor]
|
||||
if t + width > begin:
|
||||
add(tally, row, (t + width - begin) / width)
|
||||
cursor += 1
|
||||
found.append(tally)
|
||||
return found
|
||||
|
||||
|
||||
def mean(cell: list[float], at: int) -> float | None:
|
||||
return cell[at] / cell[0] if cell[0] else None
|
||||
|
||||
|
||||
def fullest(mounts: dict[str, list[float]]) -> dict:
|
||||
"""Returns:
|
||||
{value, mount, used, total} of the mount fullest on average, all
|
||||
None without a disk.
|
||||
"""
|
||||
top = max(mounts, key=lambda mount: (mean(mounts[mount], 1), mount), default=None)
|
||||
if top is None:
|
||||
return {"value": None, "mount": None, "used": None, "total": None}
|
||||
cell = mounts[top]
|
||||
return {"value": rounded(mean(cell, 1), 1), "mount": top, "used": rounded(mean(cell, 2), 0), "total": rounded(mean(cell, 3), 0)}
|
||||
|
||||
|
||||
def gauges(tally: dict) -> dict[str, dict]:
|
||||
"""Returns:
|
||||
Per gauge its mean over the rows the tally holds, each row weighted
|
||||
by its seconds inside the window: value the percent, None without a
|
||||
reading; vram adds used, the bytes, and disk names the mount fullest
|
||||
on average.
|
||||
"""
|
||||
found = {key: {"value": rounded(mean(tally[key], 1), 1)} for key in HOST}
|
||||
found["vram"] = {"value": rounded(mean(tally["vram"], 1), 1), "used": rounded(mean(tally["vram"], 2), 0)}
|
||||
found["disk"] = fullest(tally["disks"])
|
||||
return {key: {**entry, "approx": False, "source": None} for key, entry in found.items()}
|
||||
|
||||
|
||||
def rated(council: Path, kept: Path, now: float) -> dict:
|
||||
"""Args:
|
||||
council: the council directory holding the register and the score
|
||||
journal, read through scores.journal().
|
||||
kept: the directory holding the host sampler's resources folder.
|
||||
now: the end of every window in epoch seconds.
|
||||
|
||||
Returns:
|
||||
{"now", "since", "periods", "rates"}: since the first time any record
|
||||
holds; periods [{name, seconds}], total without seconds; rates per
|
||||
period per metric {value, approx, source}: the done tasks, the
|
||||
commits, the key results met and the tokens inside the window ending
|
||||
now, extrapolated() where it holds none or, for tokens, where it
|
||||
reaches back before the first token row, and the gauges() of the
|
||||
samples inside it; total spans every record.
|
||||
"""
|
||||
tasks = read_register(council / REGISTER)
|
||||
folder = kept / RESOURCES
|
||||
items = sorted(filter(None, map(placed, rows_between(folder / SAMPLES, 0, now, 0))), key=lambda item: item[0], reverse=True)
|
||||
counted = {
|
||||
"tasks": points(task_times(tasks)),
|
||||
"commits": points(commit_times(journal(council))),
|
||||
"krs": points(kr_times(tasks)),
|
||||
"tokens": token_pieces(rows_between(folder / TOKENS, 0, now, HOUR), now),
|
||||
}
|
||||
starts = {kind: [piece[0] for piece in pieces if piece[0] is not None] for kind, pieces in counted.items()}
|
||||
times = [*(t for found in starts.values() for t in found), *(item[0] for item in items)]
|
||||
since = min(times) if times else None
|
||||
length = lengths(now, since)
|
||||
begins = {"tokens": min(starts["tokens"], default=None)}
|
||||
counts = {kind: extrapolated(sums(pieces, now, FINEST.get(kind, 0)), length, held(now, since, begins.get(kind))) for kind, pieces in counted.items()}
|
||||
measured = tallies(items, [-math.inf if seconds is None else now - seconds for _, seconds in PERIODS])
|
||||
shown = {name: {**{kind: counts[kind][name] for kind in COUNTS}, **gauges(tally)} for name, tally in zip(PERIOD_NAMES, measured, strict=True)}
|
||||
return {"now": now, "since": since, "periods": [{"name": name, "seconds": seconds} for name, seconds in PERIODS], "rates": shown}
|
||||
@@ -12,6 +12,16 @@ FILE_LIMIT = 64 << 20
|
||||
|
||||
|
||||
def rows_between(path: Path, begin: float, end: float, width: float) -> list[dict]:
|
||||
"""Args:
|
||||
path: a JSON lines file of rows with t in epoch seconds.
|
||||
begin: the earliest time a row may reach to.
|
||||
end: the latest t a row may have.
|
||||
width: the seconds a row reaches past its t unless it names its own
|
||||
width.
|
||||
|
||||
Returns:
|
||||
The rows from begin to end in file order.
|
||||
"""
|
||||
try:
|
||||
lines = read_regular(path, FILE_LIMIT).splitlines()
|
||||
except (OSError, ValueError):
|
||||
@@ -19,7 +29,7 @@ def rows_between(path: Path, begin: float, end: float, width: float) -> list[dic
|
||||
found = []
|
||||
for row in map(parse_line, lines):
|
||||
t = None if row is None else epoch(row.get("t"))
|
||||
if t is not None and begin <= t + width and t <= end:
|
||||
if t is not None and t <= end and (begin <= t + width or begin <= t + (epoch(row.get("width")) or 0)):
|
||||
found.append(row)
|
||||
return found
|
||||
|
||||
|
||||
@@ -22,6 +22,21 @@ def checked(labels: object) -> dict | None:
|
||||
return labels if valid else None
|
||||
|
||||
|
||||
def journal(council: Path) -> list[dict]:
|
||||
"""Args:
|
||||
council: the council directory holding scores.jsonl.
|
||||
|
||||
Returns:
|
||||
The journal rows with a values object, in file order; none when the
|
||||
journal is missing or unreadable.
|
||||
"""
|
||||
try:
|
||||
lines = read_regular(council / SCORES, FILE_LIMIT).splitlines()
|
||||
except (OSError, ValueError):
|
||||
lines = []
|
||||
return [row for row in map(parse_line, lines) if row is not None and isinstance(row.get("values"), dict)]
|
||||
|
||||
|
||||
def scored(council: Path, skills: Path, span: tuple[float, float]) -> dict:
|
||||
"""Args:
|
||||
council: the council directory holding scores.jsonl, which score.py
|
||||
@@ -35,10 +50,5 @@ def scored(council: Path, skills: Path, span: tuple[float, float]) -> dict:
|
||||
time order.
|
||||
"""
|
||||
begin, end = span
|
||||
try:
|
||||
lines = read_regular(council / SCORES, FILE_LIMIT).splitlines()
|
||||
except (OSError, ValueError):
|
||||
lines = []
|
||||
rows = [row for row in map(parse_line, lines) if row is not None and isinstance(row.get("values"), dict)]
|
||||
inside = [row for row in rows if (moment := epoch(row.get("time"))) is not None and begin <= moment <= end]
|
||||
inside = [row for row in journal(council) if (moment := epoch(row.get("time"))) is not None and begin <= moment <= end]
|
||||
return {"since": begin, "until": end, "labels": checked(read_json(skills / LABELS)), "rows": sorted(inside, key=lambda row: row["time"])}
|
||||
@@ -9,6 +9,7 @@ from urllib.parse import urlsplit
|
||||
|
||||
from .errors import ToolError
|
||||
from .fields import FIELDS, NAME_LIMIT, expect
|
||||
from .rates import METRICS, PERIOD_NAMES
|
||||
from .records import read_json, write_atomic
|
||||
|
||||
THEMES = ("sky", "light", "dark", "custom")
|
||||
@@ -43,7 +44,9 @@ SETTINGS = {
|
||||
"estimation": "none",
|
||||
"trend": {"mode": "off", "days": 7},
|
||||
"bubbles": False,
|
||||
"rates": {"period": "hour", "metrics": ["tasks", "tokens", "cpu"]},
|
||||
}
|
||||
RATE_CHIP = (2, 3)
|
||||
SPACE_AXES = ("none", "time", "spawn")
|
||||
SPACE_LIMITS = {
|
||||
"graph": {"yaw": (-180.0, 180.0), "pitch": (-90.0, 90.0), "zoom": (0.25, 4.0), "pan_x": (-2000.0, 2000.0), "pan_y": (-2000.0, 2000.0)},
|
||||
@@ -172,6 +175,17 @@ def trend_setting(raw: object) -> dict:
|
||||
return {"mode": entry["mode"], "days": entry["days"]}
|
||||
|
||||
|
||||
def rates_setting(raw: object) -> dict:
|
||||
entry = section(raw, "rates", tuple(SETTINGS["rates"]))
|
||||
low, high = RATE_CHIP
|
||||
metrics = entry["metrics"]
|
||||
if not isinstance(entry["period"], str) or entry["period"] not in PERIOD_NAMES:
|
||||
raise ToolError(f"rates.period must be one of {', '.join(PERIOD_NAMES)}")
|
||||
if not isinstance(metrics, list) or not low <= len(metrics) <= high or not all(isinstance(metric, str) and metric in METRICS for metric in metrics) or len(set(metrics)) != len(metrics):
|
||||
raise ToolError(f"rates.metrics must list {low} to {high} distinct of {', '.join(METRICS)}")
|
||||
return {"period": entry["period"], "metrics": list(metrics)}
|
||||
|
||||
|
||||
def valid_settings(raw: object) -> dict:
|
||||
data = section(raw, "settings", tuple(SETTINGS))
|
||||
colors = section(data["colors"], "colors", tuple(SETTINGS["colors"]))
|
||||
@@ -195,9 +209,18 @@ def valid_settings(raw: object) -> dict:
|
||||
"estimation": data["estimation"],
|
||||
"trend": trend_setting(data["trend"]),
|
||||
"bubbles": flag(data["bubbles"], "bubbles"),
|
||||
"rates": rates_setting(data["rates"]),
|
||||
}
|
||||
|
||||
|
||||
def completed(raw: object, saved: dict) -> object:
|
||||
"""Returns:
|
||||
raw with the saved rates when it names none, as a page loaded before
|
||||
the rates chip existed sends it; anything else unchanged.
|
||||
"""
|
||||
return {**raw, "rates": saved["rates"]} if isinstance(raw, dict) and "rates" not in raw else raw
|
||||
|
||||
|
||||
def radio_moved(stored: object) -> object:
|
||||
"""Returns:
|
||||
The stored settings without the tones, the squelch and the read-aloud
|
||||
|
||||
@@ -36,7 +36,7 @@ from .register import (
|
||||
stored_order,
|
||||
)
|
||||
from .reviewing import review_row
|
||||
from .settings import media_origins, station_origins, valid_settings
|
||||
from .settings import completed, media_origins, station_origins, valid_settings
|
||||
from .signed import signing
|
||||
from .skill_scripts import OUTPUT_LIMIT
|
||||
from .tasks import queue_chain, queue_line, task_edges, task_rows
|
||||
@@ -206,7 +206,8 @@ class View:
|
||||
return {**settings, "streams": streams}, problems
|
||||
|
||||
def sign_settings(self, body: bytes) -> dict:
|
||||
return self.challenge_for(self.signed_changes(valid_settings(decode(body)), self.settings.get()))
|
||||
saved = self.settings.get()
|
||||
return self.challenge_for(self.signed_changes(valid_settings(completed(decode(body), saved)), saved))
|
||||
|
||||
def save_settings(self, body: bytes) -> dict:
|
||||
"""Returns:
|
||||
@@ -216,8 +217,8 @@ class View:
|
||||
"""
|
||||
payload = decode(body)
|
||||
proof = payload.pop("proof", None)
|
||||
settings = valid_settings(payload)
|
||||
saved = self.settings.get()
|
||||
settings = valid_settings(completed(payload, saved))
|
||||
for change in self.signed_changes(settings, saved):
|
||||
self.operator_proven(proof, change)
|
||||
resolved, problems = self.resolved(settings, saved)
|
||||
|
||||
@@ -415,6 +415,8 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
|
||||
{% include "style-resources.css" %}
|
||||
|
||||
{% include "style-rates.css" %}
|
||||
|
||||
{% include "style-legend-chips.css" %}
|
||||
|
||||
{% include "style-inbox.css" %}
|
||||
@@ -454,6 +456,7 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
<span class="spacer"></span>
|
||||
<details class="menu" id="time"></details>
|
||||
<span class="updated" id="updated" aria-live="polite"></span>
|
||||
<div class="rates" id="rates"></div>
|
||||
</div>
|
||||
<div class="banner" id="mayday" role="alert" hidden></div>
|
||||
</div>
|
||||
@@ -611,6 +614,8 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
|
||||
{% include "page-resources.js" %}
|
||||
|
||||
{% include "page-rates.js" %}
|
||||
|
||||
{% include "page-home.js" %}
|
||||
|
||||
{% include "page-rules.js" %}
|
||||
|
||||
@@ -66,6 +66,7 @@
|
||||
const updated = document.getElementById("updated");
|
||||
updated.classList.toggle("offline", ui.offline);
|
||||
updated.textContent = `${ui.offline ? "No connection · " : ""}${live() ? "Live" : "As of"} ${absolute.format(stampOf(live() ? state.now : state.until))}`;
|
||||
pollRates();
|
||||
}
|
||||
|
||||
const faceParts = (key) => Faces.of(key).parts.map(([tag, attrs]) => s(tag, attrs));
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
const RATES = {poll: 60, fewest: 2, most: 3};
|
||||
const RATE_PERIODS = [
|
||||
["minute", "min", "per minute"],
|
||||
["hour", "h", "per hour"],
|
||||
["day", "d", "per day"],
|
||||
["week", "wk", "per week"],
|
||||
["month", "mo", "per month of 30 days"],
|
||||
["year", "yr", "per year of 365 days"],
|
||||
["total", "all", "in total since records begin"],
|
||||
];
|
||||
const RATE_METRICS = [
|
||||
["tasks", "bi-check2-square", "Tasks done", "count"],
|
||||
["commits", "bi-git", "Commits", "count"],
|
||||
["krs", "bi-bullseye", "Key results met", "count"],
|
||||
["tokens", "bi-coin", "Tokens", "count"],
|
||||
["cpu", "bi-cpu", "CPU", "percent"],
|
||||
["ram", "bi-memory", "RAM", "percent"],
|
||||
["vram", "bi-gpu-card", "VRAM", "vram"],
|
||||
["swap", "bi-arrow-left-right", "Swap", "percent"],
|
||||
["disk", "bi-hdd", "Disk", "disk"],
|
||||
];
|
||||
const BYTE_UNITS = ["B", "KiB", "MiB", "GiB", "TiB", "PiB"];
|
||||
const rateView = {data: null, dom: null, busy: false, at: -Infinity};
|
||||
const periodOf = (id) => RATE_PERIODS.find(([name]) => name === id) || RATE_PERIODS[1];
|
||||
const metricOf = (id) => RATE_METRICS.find(([name]) => name === id);
|
||||
|
||||
function bytesText(value) {
|
||||
const power = value > 0 ? Math.min(Math.floor(Math.log2(value) / 10), BYTE_UNITS.length - 1) : 0;
|
||||
return `${power ? fixed(value / 2 ** (10 * power)) : Math.round(value)} ${BYTE_UNITS[power]}`;
|
||||
}
|
||||
|
||||
function countText(value) {
|
||||
if (value >= 1000) {
|
||||
return compactCount(value);
|
||||
}
|
||||
if (value >= 10 || Number.isInteger(value)) {
|
||||
return String(Math.round(value));
|
||||
}
|
||||
return value < 0.1 ? String(Number(value.toPrecision(1))) : fixed(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param entry - one metric of one period from /api/rates, or null before the first answer.
|
||||
* @param kind - the metric's kind: count, percent, vram or disk.
|
||||
* @param short - the period's short name a count is per, "" for none.
|
||||
* @returns the cell text: ≈ before an extrapolated count, n/a for a gauge without a sample.
|
||||
*/
|
||||
function rateText(entry, kind, short) {
|
||||
if (!entry) {
|
||||
return "…";
|
||||
}
|
||||
if (entry.value === null) {
|
||||
return "n/a";
|
||||
}
|
||||
if (kind === "count") {
|
||||
return `${entry.approx ? "≈ " : ""}${countText(entry.value)}${short ? `/${short}` : ""}`;
|
||||
}
|
||||
return kind === "vram" && entry.used !== null ? `${percentOf(entry.value)} · ${bytesText(entry.used)}` : percentOf(entry.value);
|
||||
}
|
||||
|
||||
function rateHint(entry, metric, period) {
|
||||
const [id, , label, kind] = metric;
|
||||
const [, , per] = period;
|
||||
if (!entry) {
|
||||
return `${label} ${per}: loading`;
|
||||
}
|
||||
if (entry.value === null) {
|
||||
return `${label} ${per}: n/a, no sample in this window`;
|
||||
}
|
||||
const source = entry.source === "total" ? "all records of this kind" : `the last ${entry.source}`;
|
||||
const scaled = entry.approx ? `; ≈ none in this window or its records begin later, scaled from ${source}` : "";
|
||||
const where = id === "disk" ? ` on ${entry.mount}, ${bytesText(entry.used)} of ${bytesText(entry.total)}` : "";
|
||||
const mean = kind === "count" ? "" : " on average";
|
||||
return `${label} ${per}: ${rateText(entry, kind, "")}${mean}${where}${scaled}`;
|
||||
}
|
||||
|
||||
function ratesBar() {
|
||||
const host = document.getElementById("rates");
|
||||
const period = h("select", {class: "field rates-period", ...named("Rate period", "Rate period: the window the chip counts over"), on: {change: (event) => chooseRates({...settings.rates, period: event.target.value})}},
|
||||
RATE_PERIODS.map(([id, short, per]) => h("option", {value: id, title: per}, short)));
|
||||
const chip = h("summary", {class: "rates-chip"});
|
||||
const body = h("tbody", {});
|
||||
const head = h("tr", {}, h("th", {scope: "col"}, "Metric"), h("th", {scope: "col", title: "Shown in the chip"}, "Chip"), RATE_PERIODS.map(([id, short, per]) => h("th", {scope: "col", "data-period": id, title: per}, short)));
|
||||
const box = h("div", {class: "menu-box end rates-box", role: "group", "aria-label": "Rates per period"},
|
||||
h("h3", {}, "Rates per period"),
|
||||
h("div", {class: "rates-scroll"}, h("table", {class: "rates-table"}, h("thead", {}, head), body)),
|
||||
h("p", {class: "muted"}, `≈ means an estimate: none in the window, so the shortest longer window that holds any, scaled down, or a window reaching back before the records of its kind begin, scaled from those records. n/a means no sample in the window. Total counts since records begin; the gauges average over the window. The chip shows ${RATES.fewest} to ${RATES.most} metrics.`));
|
||||
const menu = h("details", {class: "menu rates-menu"}, chip, box);
|
||||
host.replaceChildren(period, menu);
|
||||
rateView.dom = {period, chip, body, menu};
|
||||
}
|
||||
|
||||
function rateCell(metric, period, chosen) {
|
||||
const entry = rateView.data ? rateView.data.rates[period[0]][metric[0]] : null;
|
||||
const short = period[0] === "total" ? "" : period[1];
|
||||
return h("td", {"data-period": period[0], "data-approx": String(Boolean(entry && entry.approx)), "aria-current": period[0] === chosen.period ? "true" : null, title: rateHint(entry, metric, period)},
|
||||
rateText(entry, metric[3], metric[3] === "count" ? short : ""));
|
||||
}
|
||||
|
||||
function rateRow(metric, chosen) {
|
||||
const [id, name, label] = metric;
|
||||
const shown = chosen.metrics.includes(id);
|
||||
const locked = shown ? chosen.metrics.length <= RATES.fewest : chosen.metrics.length >= RATES.most;
|
||||
const pick = h("input", {type: "checkbox", checked: shown, disabled: locked, "aria-label": `${label} in the chip`, on: {change: () => chooseRates({...chosen, metrics: shown ? chosen.metrics.filter((other) => other !== id) : [...chosen.metrics, id]})}});
|
||||
return h("tr", {"data-metric": id}, h("th", {scope: "row"}, icon(name), ` ${label}`), h("td", {}, pick), RATE_PERIODS.map((period) => rateCell(metric, period, chosen)));
|
||||
}
|
||||
|
||||
function drawRates() {
|
||||
const dom = rateView.dom;
|
||||
if (!dom) {
|
||||
return;
|
||||
}
|
||||
const chosen = settings.rates;
|
||||
const period = periodOf(chosen.period);
|
||||
const short = period[0] === "total" ? "" : period[1];
|
||||
const entries = chosen.metrics.map((id) => [metricOf(id), rateView.data ? rateView.data.rates[period[0]][id] : null]);
|
||||
dom.period.value = period[0];
|
||||
dom.chip.replaceChildren(...entries.map(([metric, entry]) => h("span", {class: "rate", "data-metric": metric[0], "data-approx": String(Boolean(entry && entry.approx)), title: rateHint(entry, metric, period)},
|
||||
icon(metric[1]), h("span", {class: "rate-value"}, rateText(entry, metric[3], metric[3] === "count" ? short : "")))), caret());
|
||||
dom.chip.setAttribute("aria-label", `Rates ${period[2]}: ${entries.map(([metric, entry]) => `${metric[2]} ${rateText(entry, metric[3], "")}`).join(", ")}`);
|
||||
dom.chip.title = "Rates: every metric per period";
|
||||
const focused = document.activeElement instanceof Element ? document.activeElement.closest("#rates tr[data-metric]") : null;
|
||||
dom.body.replaceChildren(...RATE_METRICS.map((metric) => rateRow(metric, chosen)));
|
||||
const again = focused ? dom.body.querySelector(`tr[data-metric="${focused.dataset.metric}"] input`) : null;
|
||||
if (again) {
|
||||
again.focus();
|
||||
}
|
||||
}
|
||||
|
||||
async function chooseRates(next) {
|
||||
if (!(await saveSettings({...settings, rates: next}))) {
|
||||
drawRates();
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchRates() {
|
||||
rateView.busy = true;
|
||||
try {
|
||||
const response = await fetch("/api/rates", {cache: "no-store"});
|
||||
if (response.ok) {
|
||||
rateView.data = await response.json();
|
||||
}
|
||||
} catch {
|
||||
ui.offline = true;
|
||||
}
|
||||
rateView.at = nowSeconds();
|
||||
rateView.busy = false;
|
||||
drawRates();
|
||||
}
|
||||
|
||||
function pollRates() {
|
||||
if (!rateView.dom) {
|
||||
ratesBar();
|
||||
drawRates();
|
||||
}
|
||||
if (!document.hidden && !rateView.busy && nowSeconds() - rateView.at >= RATES.poll) {
|
||||
fetchRates();
|
||||
}
|
||||
}
|
||||
@@ -9,7 +9,7 @@
|
||||
const RESOURCE_BURN = {done: "Done", scope: "Scope"};
|
||||
const resourceView = {data: null, dom: null, busy: false, key: "", at: 0, timer: 0};
|
||||
const resourceKey = () => JSON.stringify([ui.preset, ui.from, ui.to, ui.asOf]);
|
||||
const compactCount = (value) => (value >= 1e6 ? `${fixed(value / 1e6)}M` : value >= 1e3 ? `${fixed(value / 1e3)}k` : String(Math.round(value)));
|
||||
const compactCount = Resources.compact;
|
||||
const resourceLabel = (line) => (line.kind === "vram" ? `${line.name} VRAM` : line.name);
|
||||
const percentOf = (value) => `${value} %`;
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
syncRadio();
|
||||
musicButton();
|
||||
syncBubbles();
|
||||
drawRates();
|
||||
}
|
||||
|
||||
async function saveSettings(next) {
|
||||
@@ -153,6 +154,7 @@
|
||||
estimation: settings.estimation,
|
||||
trend: settings.trend,
|
||||
bubbles: settings.bubbles,
|
||||
rates: settings.rates,
|
||||
};
|
||||
if (await saveSettings(next)) {
|
||||
closeSheet();
|
||||
|
||||
@@ -8,6 +8,8 @@ const Resources = (() => {
|
||||
const HOUR = 3600;
|
||||
const COUNTS = ["input", "output", "cache_read", "cache_write"];
|
||||
const ZONES = [{kind: "idle", from: 0, to: IDLE}, {kind: "saturated", from: SATURATED, to: 100}];
|
||||
const COUNT_UNITS = [[1e12, "T"], [1e9, "B"], [1e6, "M"], [1e3, "k"]];
|
||||
const ROUNDS_UP = 0.99995;
|
||||
const finite = (value) => typeof value === "number" && Number.isFinite(value);
|
||||
const share = (part, whole) => (finite(part) && finite(whole) && whole > 0 ? Math.round((1000 * part) / whole) / 10 : null);
|
||||
const listOf = (row, key) => (Array.isArray(row[key]) ? row[key].filter((item) => item && typeof item === "object") : []);
|
||||
@@ -178,7 +180,17 @@ const Resources = (() => {
|
||||
return {top, done: points.map((point) => [point.t, percent(point.done)]), scope: points.map((point) => [point.t, percent(point.scope)])};
|
||||
}
|
||||
|
||||
return {IDLE, SATURATED, CALM, GAP, HOUR, COUNTS, ZONES, lines, hardware, segments, spans, waits, asleep, unknown, stack, accumulate, burnup};
|
||||
/**
|
||||
* @param value - a count.
|
||||
* @returns the count with one decimal in the largest of k, M, B and T it reaches once rounded to that decimal,
|
||||
* else as a whole number.
|
||||
*/
|
||||
function compact(value) {
|
||||
const unit = COUNT_UNITS.find(([size]) => value >= size * ROUNDS_UP);
|
||||
return unit ? `${(value / unit[0]).toFixed(1)}${unit[1]}` : String(Math.round(value));
|
||||
}
|
||||
|
||||
return {IDLE, SATURATED, CALM, GAP, HOUR, COUNTS, ZONES, lines, hardware, segments, spans, waits, asleep, unknown, stack, accumulate, burnup, compact};
|
||||
})();
|
||||
if (typeof module === "object" && module.exports) {
|
||||
module.exports = Resources;
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
.rates { display: inline-flex; align-items: center; gap: 4px; min-width: 0; max-width: 100%; }
|
||||
.rates-period { padding: 3px 6px; border-radius: 999px; font-size: var(--fs-xs); }
|
||||
.rates-menu > summary.rates-chip { gap: 10px; padding: 3px 10px; border-radius: 999px; font-size: var(--fs-sm); font-weight: 560; font-variant-numeric: tabular-nums; white-space: nowrap; }
|
||||
.rates-chip .rate { display: inline-flex; align-items: center; gap: 4px; }
|
||||
.rates-chip .bi { color: var(--muted); }
|
||||
.rates-chip .rate[data-approx="true"] .rate-value, .rates-table td[data-approx="true"] { color: var(--muted); }
|
||||
.rates-box { max-width: min(1100px, calc(100vw - 32px)); }
|
||||
.rates-scroll { max-width: 100%; overflow-x: auto; }
|
||||
.rates-box > p { width: 0; min-width: 100%; margin: 0; }
|
||||
.rates-table { border-collapse: collapse; font-size: var(--fs-sm); font-variant-numeric: tabular-nums; }
|
||||
.rates-table th, .rates-table td { padding: 3px 8px; border-bottom: 1px solid var(--line); text-align: right; white-space: nowrap; }
|
||||
.rates-table thead th { color: var(--muted); font-size: var(--fs-xs); font-weight: 650; }
|
||||
.rates-table th[scope="row"], .rates-table thead th:first-child { position: sticky; left: 0; z-index: 1; background: var(--panel); font-weight: 560; text-align: left; }
|
||||
.rates-table td[aria-current="true"] { background: var(--accent-soft); color: var(--accent); }
|
||||
@media (min-width: 641px) {
|
||||
.rates-box { width: max-content; }
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
.rates-menu > summary.rates-chip { gap: 6px; padding: 3px 8px; }
|
||||
}
|
||||
@@ -3,7 +3,11 @@
|
||||
Each `--message-block` render appends one row to
|
||||
`${AGENTS_COUNCIL_ROOT:-/tmp/agents-council}/scores.jsonl`: time, origin
|
||||
remote, top level path, branch, `sha` null, the six values and the block's
|
||||
digest. The hub's burndown draws it as overlays; no gate reads it. A failed
|
||||
digest. The commit table after the commit (`--commit-table <repo> --commit
|
||||
<sha>`) appends the same row for that repository with the full sha and the
|
||||
commit time, unless the journal holds one for that remote and sha already;
|
||||
the hub counts each such row once as a commit. The hub's burndown draws the
|
||||
rows as overlays; no gate reads them. A failed
|
||||
append prints one stderr line and keeps the exit code; a linked council
|
||||
root or a link, FIFO or other non-regular file at the journal fails it
|
||||
without blocking, through `nofollow.py` of `kvb-org-organize`; when that
|
||||
|
||||
@@ -21,7 +21,6 @@ Exit codes:
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from collections import deque
|
||||
@@ -34,9 +33,9 @@ from journal import (
|
||||
entry,
|
||||
git_read,
|
||||
journal_path,
|
||||
known,
|
||||
load_labels,
|
||||
origin,
|
||||
read,
|
||||
)
|
||||
|
||||
FIELD = "\x00"
|
||||
@@ -185,22 +184,6 @@ def scan(repo: Path, since: str | None, named: dict[str, str], extras: set[str],
|
||||
return place[1], rows
|
||||
|
||||
|
||||
def known(path: Path) -> set[tuple[str | None, str]]:
|
||||
try:
|
||||
text = read(path)
|
||||
except FileNotFoundError:
|
||||
return set()
|
||||
keys = set()
|
||||
for line in text.splitlines():
|
||||
try:
|
||||
row = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if isinstance(row, dict) and isinstance(row.get("sha"), str):
|
||||
keys.add((row.get("remote"), row["sha"]))
|
||||
return keys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--repo", type=Path, nargs="+", action="extend", required=True)
|
||||
|
||||
@@ -182,3 +182,25 @@ def read(path: Path) -> str:
|
||||
Its text, undecodable bytes replaced.
|
||||
"""
|
||||
return nofollow(lambda helper: helper.read_text(path.parent, path))
|
||||
|
||||
|
||||
def known(path: Path) -> set[tuple[str | None, str]]:
|
||||
"""Args:
|
||||
path: the journal, read through read(); a missing one holds none.
|
||||
|
||||
Returns:
|
||||
(remote, sha) of every row with a sha.
|
||||
"""
|
||||
try:
|
||||
text = read(path)
|
||||
except FileNotFoundError:
|
||||
return set()
|
||||
keys = set()
|
||||
for line in text.splitlines():
|
||||
try:
|
||||
row = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if isinstance(row, dict) and isinstance(row.get("sha"), str):
|
||||
keys.add((row.get("remote"), row["sha"]))
|
||||
return keys
|
||||
@@ -17,7 +17,9 @@ Args:
|
||||
that repository, then a blank line, `📁 Worktree:` and the top level path
|
||||
alone in a ```text block, so it copies in one go.
|
||||
--commit <sha>: with --commit-table, the diff stat of that commit instead of
|
||||
the staged one, plus a commit column.
|
||||
the staged one, plus a commit column; appends one row with the full sha
|
||||
and the commit time to the score journal of journal.py unless it holds
|
||||
one for that remote and sha, so the hub counts the commit once.
|
||||
--message-block: print the commit message block instead of the two lines, one
|
||||
value per line: `🎯 Fix: <f> %`, `💥 Break: <b> %`, `💸 Tokens: <c>`,
|
||||
`⏱️ Velocity: <t>`, `🛡️ Security: <n> %`, `💰 Value: <n:n>` with --value only,
|
||||
@@ -61,6 +63,7 @@ from journal import (
|
||||
git,
|
||||
git_read,
|
||||
journal_path,
|
||||
known,
|
||||
load_labels,
|
||||
origin,
|
||||
)
|
||||
@@ -258,6 +261,10 @@ def main() -> int:
|
||||
if args.value:
|
||||
header.insert(list(metrics).index("tokens") + 1, value_header)
|
||||
row.insert(list(metrics).index("tokens") + 1, args.value)
|
||||
lines = [f"{metric['emoji']} {metric['label']}: {measured[key]}" for key, metric in metrics.items()]
|
||||
if args.value:
|
||||
lines.insert(list(metrics).index("score"), f"{value_header}: {args.value}")
|
||||
block = "\n".join((*lines, f"{extras['gate']['emoji']} {extras['gate']['label']}: {verdict_cell}"))
|
||||
if args.commit_table:
|
||||
try:
|
||||
lead, cells = commit_cells(args.commit_table, args.commit)
|
||||
@@ -266,15 +273,12 @@ def main() -> int:
|
||||
print(f"score: {error}", file=sys.stderr)
|
||||
return 4 if code == 4 else 2
|
||||
print(table([*lead, *header], [*cells, *row]))
|
||||
print(f"\n{WORKTREE}\n\n```text\n{top}\n```")
|
||||
print(f"\n{WORKTREE}\n\n```text\n{top}\n```", flush=True)
|
||||
if args.commit:
|
||||
committed(args.commit_table, args.commit, values_of(args, total), block)
|
||||
elif args.table:
|
||||
print(table(header, row))
|
||||
elif args.message_block:
|
||||
lines = [f"{metric['emoji']} {metric['label']}: {measured[key]}" for key, metric in metrics.items()]
|
||||
if args.value:
|
||||
lines.insert(list(metrics).index("score"), f"{value_header}: {args.value}")
|
||||
gate_label = extras["gate"]
|
||||
block = "\n".join((*lines, f"{gate_label['emoji']} {gate_label['label']}: {verdict_cell}"))
|
||||
print(block, flush=True)
|
||||
journal(args, total, block)
|
||||
else:
|
||||
@@ -289,18 +293,44 @@ def plain(value: float | str) -> float | int | None:
|
||||
return int(value) if value.is_integer() else value
|
||||
|
||||
|
||||
def journal(args: argparse.Namespace, total: int, block: str) -> None:
|
||||
values = {
|
||||
def values_of(args: argparse.Namespace, total: int) -> dict:
|
||||
return {
|
||||
"fix": plain(args.fix), "break": plain(args.risk), "tokens": plain(args.efficiency),
|
||||
"velocity": plain(args.speed), "security": plain(args.security[0]), "score": total,
|
||||
}
|
||||
|
||||
|
||||
def journal(args: argparse.Namespace, total: int, block: str) -> None:
|
||||
path = journal_path()
|
||||
try:
|
||||
here = Path.cwd()
|
||||
append(path, [entry(int(time.time()), origin(here), git_read(here, "branch", "--show-current") or None, None, values, block)])
|
||||
append(path, [entry(int(time.time()), origin(here), git_read(here, "branch", "--show-current") or None, None, values_of(args, total), block)])
|
||||
except (OSError, ValueError) as error:
|
||||
print(f"score: journal {path}: {getattr(error, 'strerror', None) or error}", file=sys.stderr)
|
||||
|
||||
|
||||
def committed(repo: str, commit: str, values: dict, block: str) -> None:
|
||||
"""Args:
|
||||
repo: the --commit-table repository.
|
||||
commit: the --commit revision.
|
||||
values: values_of() the block.
|
||||
block: the message block the commit carries.
|
||||
|
||||
Returns:
|
||||
Appends one row with the full sha and the commit time unless the
|
||||
journal holds one for its remote and sha; a failure prints one
|
||||
stderr line.
|
||||
"""
|
||||
path = journal_path()
|
||||
try:
|
||||
sha = git(repo, "rev-parse", "--verify", f"{commit}^{{commit}}")
|
||||
place = origin(Path(repo))
|
||||
if (place[0], sha) not in known(path):
|
||||
when = int(git(repo, "show", "-s", "--format=%ct", sha))
|
||||
append(path, [entry(when, place, git_read(Path(repo), "branch", "--show-current") or None, sha, values, block)])
|
||||
except (OSError, ValueError, LookupError) as error:
|
||||
print(f"score: journal {path}: {getattr(error, 'strerror', None) or error}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -449,8 +449,13 @@ def cmd_kr(args: argparse.Namespace) -> int:
|
||||
index = int(args.value) - 1
|
||||
if args.action == "rm":
|
||||
results.pop(index)
|
||||
elif args.action == "met":
|
||||
if results[index].get("met") is not True:
|
||||
results[index]["met_at"] = stamp()
|
||||
results[index]["met"] = True
|
||||
else:
|
||||
results[index]["met"] = args.action == "met"
|
||||
results[index]["met"] = False
|
||||
results[index].pop("met_at", None)
|
||||
if not results:
|
||||
entry.pop("key_results")
|
||||
print(f"{args.task}: key result {args.action} {args.value}")
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path[:0] = [str(HERE.parent), str(HERE)]
|
||||
|
||||
import test_orchestrator_e2e as base
|
||||
import test_workbench_w4_e2e as w4
|
||||
from hub_case import HubCase
|
||||
from hub_support import iso
|
||||
|
||||
OUT = Path(os.environ.get("E2E_OUT", "/tmp/hub-header-rates-shots"))
|
||||
HOUR = 3600
|
||||
GIB = 1 << 30
|
||||
PERIODS = {1920: "day", 1366: "week", 390: "minute"}
|
||||
BEFORE = {1920: "hour", 1366: "day", 390: "week"}
|
||||
CHOSEN = {1920: ["tasks", "tokens", "commits"], 1366: ["tasks", "tokens", "cpu"], 390: ["tasks", "tokens", "commits"]}
|
||||
TASKS = {"minute": ("≈ 0.02/min", "true"), "hour": ("1/h", "false"), "day": ("2/d", "false"), "week": ("2/wk", "false"), "month": ("2/mo", "false"), "year": ("2/yr", "false"), "total": ("2", "false")}
|
||||
METRICS = ["tasks", "commits", "krs", "tokens", "cpu", "ram", "vram", "swap", "disk"]
|
||||
HEAD = ["Metric", "Chip", "min", "h", "d", "wk", "mo", "yr", "all"]
|
||||
READ = """
|
||||
const chip = () => document.querySelector("#rates summary.rates-chip");
|
||||
const period = () => document.querySelector('#rates select[aria-label="Rate period"]');
|
||||
const menu = () => document.querySelector("#rates details.rates-menu");
|
||||
const box = () => document.querySelector("#rates .rates-box");
|
||||
const pick = (id) => document.querySelector(`#rates tr[data-metric="${id}"] input[type="checkbox"]`);
|
||||
const metrics = () => all("#rates summary .rate").map((node) => node.dataset.metric);
|
||||
const values = () => Object.fromEntries(all("#rates summary .rate").map((node) => [node.dataset.metric, text(node)]));
|
||||
const loaded = () => Boolean(chip() && metrics().length && !text(chip()).includes("…"));
|
||||
const stored = async () => (await (await fetch("/api/state", {cache: "no-store"})).json()).settings.rates;
|
||||
const storedUntil = async (test, ms) => {
|
||||
const end = Date.now() + ms;
|
||||
while (Date.now() < end) {
|
||||
const found = await stored();
|
||||
if (test(found)) {
|
||||
return found;
|
||||
}
|
||||
await sleep(200);
|
||||
}
|
||||
return null;
|
||||
};
|
||||
const inside = (node) => {
|
||||
const rect = node.getBoundingClientRect();
|
||||
return rect.width > 0 && rect.left >= -1 && rect.right <= innerWidth + 1;
|
||||
};
|
||||
const fits = () => {
|
||||
const bar = document.querySelector(".navbar");
|
||||
return {wide: document.documentElement.scrollWidth > innerWidth + 1, bar: bar.scrollWidth > bar.clientWidth + 1, chip: inside(chip()), period: inside(period())};
|
||||
};
|
||||
"""
|
||||
CHOOSE = """(async () => {@HELPERS@@READ@
|
||||
if (!(await until(loaded, 15000))) {
|
||||
return {error: text(document.getElementById("rates"))};
|
||||
}
|
||||
const before = {period: period().value, metrics: metrics()};
|
||||
period().value = "@PERIOD@";
|
||||
period().dispatchEvent(new Event("change", {bubbles: true}));
|
||||
const saved = await storedUntil((found) => found.period === "@PERIOD@", 8000);
|
||||
await until(() => period().value === "@PERIOD@" && loaded(), 4000);
|
||||
return {before, saved, after: {period: period().value, metrics: metrics(), values: values(), label: chip().getAttribute("aria-label")}, fits: fits(), open: menu().open, violations: window.e2eViolations || null};
|
||||
})()"""
|
||||
OPEN = """(async () => {@HELPERS@@READ@
|
||||
if (!(await until(loaded, 15000))) {
|
||||
return {error: text(document.getElementById("rates"))};
|
||||
}
|
||||
const kept = {period: period().value, metrics: metrics(), values: values()};
|
||||
chip().click();
|
||||
const opened = Boolean(await until(() => menu().open && box().getClientRects().length, 4000));
|
||||
const head = all("#rates .rates-table thead th").map(text);
|
||||
const rows = all("#rates .rates-table tbody tr").map((row) => [row.dataset.metric, all("td[data-period]", row).length]);
|
||||
const current = all('#rates .rates-table td[aria-current="true"]').map((cell) => cell.dataset.period);
|
||||
const tasks = Object.fromEntries(all('#rates tr[data-metric="tasks"] td[data-period]').map((cell) => [cell.dataset.period, [text(cell), cell.dataset.approx]]));
|
||||
const hint = document.querySelector('#rates tr[data-metric="tasks"] td[data-period="minute"]').title;
|
||||
const [drop, add] = metrics().includes("cpu") ? ["cpu", "commits"] : ["commits", "cpu"];
|
||||
const locked = pick(add).disabled;
|
||||
pick(drop).click();
|
||||
await storedUntil((found) => !found.metrics.includes(drop), 8000);
|
||||
await until(() => !metrics().includes(drop) && pick(add) && !pick(add).disabled, 4000);
|
||||
const fewest = all("#rates tbody input:checked").map((input) => input.disabled);
|
||||
pick(add).click();
|
||||
const swapped = await storedUntil((found) => found.metrics.includes(add), 8000);
|
||||
await until(() => metrics().includes(add), 4000);
|
||||
return {kept, opened, head, rows, current, tasks, hint, locked, fewest, swapped, chip: metrics(), box: inside(box()), fits: fits(), open: menu().open, violations: window.e2eViolations || null};
|
||||
})()"""
|
||||
CLOSE = """(async () => {@HELPERS@@READ@
|
||||
const was = menu().open;
|
||||
document.body.dispatchEvent(new KeyboardEvent("keydown", {key: "Escape", bubbles: true}));
|
||||
const escaped = Boolean(await until(() => !menu().open, 3000));
|
||||
chip().click();
|
||||
const reopened = Boolean(await until(() => menu().open, 3000));
|
||||
chip().click();
|
||||
const clicked = Boolean(await until(() => !menu().open, 3000));
|
||||
chip().click();
|
||||
await until(() => menu().open, 3000);
|
||||
document.getElementById("main").click();
|
||||
const outside = Boolean(await until(() => !menu().open, 3000));
|
||||
return {was, escaped, reopened, clicked, outside, fits: fits(), violations: window.e2eViolations || null};
|
||||
})()"""
|
||||
|
||||
|
||||
def probe(template: str, width: int) -> str:
|
||||
names = {"@HELPERS@": base.HELPERS, "@READ@": READ, "@PERIOD@": PERIODS[width]}
|
||||
for key, value in names.items():
|
||||
template = template.replace(key, value)
|
||||
return template
|
||||
|
||||
|
||||
def sample(t: int) -> dict:
|
||||
gpus = [{"name": "0000:01:00.0", "util": 30.0, "vram_used": 2 * GIB, "vram_total": 8 * GIB}]
|
||||
return {"t": t, "disks": [{"mount": "/", "used": 400 << 30, "total": 1000 << 30}], "cpu": 25.0, "ram": 40.0, "swap": 5.0, "load": 1.0, "gpus": gpus, "queue": {"waiting": 0, "active": 0}}
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which(os.environ.get("CHROMIUM", "chromium")), "no Chromium")
|
||||
class TestRatesChip(HubCase):
|
||||
drive = base.TestOrchestratorE2E.drive
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
now = int(time.time())
|
||||
hour = now - now % HOUR
|
||||
folder = self.kept / "resources"
|
||||
folder.mkdir()
|
||||
(folder / "samples.jsonl").write_text(self.jsonl([sample(now - 2 * HOUR + step * 60) for step in range(140)]), encoding="utf-8")
|
||||
tokens = [{"t": t, "agent": "Skills", "model": "claude-opus-5-5", "input": 4000, "output": 1000, "cache_read": 20000, "cache_write": 500} for t in (hour - HOUR, hour)]
|
||||
(folder / "tokens.jsonl").write_text(self.jsonl(tokens), encoding="utf-8")
|
||||
self.write_council({"rasci.json": {
|
||||
"work/first": {"title": "First", "status": "done", "worker": "Skills", "start": iso(now - 5 * HOUR), "end": iso(now - 3 * HOUR), "roles": {"Skills": "RA"}},
|
||||
"work/second": {"title": "Second", "status": "done", "worker": "Skills", "start": iso(now - 2 * HOUR), "end": iso(now - 1200), "roles": {"Skills": "RA"}, "key_results": [{"text": "shipped", "met": True, "met_at": iso(now - 1300)}]},
|
||||
"work/third": {"title": "Third", "status": "running", "worker": "Stage", "start": iso(now - HOUR), "roles": {"Stage": "RA"}},
|
||||
}})
|
||||
(self.council / "scores.jsonl").write_text(json.dumps({"time": now - 2 * HOUR, "remote": "example.org/skills", "path": "/repo", "branch": "main", "sha": "d" * 40, "values": {"score": 90}, "digest": "0" * 64}) + "\n", encoding="utf-8")
|
||||
self.base = f"http://{self.host}/"
|
||||
|
||||
def steps(self, browser: str) -> list[dict]:
|
||||
steps = []
|
||||
for width, height in base.WIDTHS:
|
||||
shot = OUT / f"{browser}-{width}"
|
||||
steps += [
|
||||
{"width": width, "height": height, "url": f"{self.base}?{width}#home", "wait": 1200, "probe": probe(CHOOSE, width), "shot": f"{shot}-chip.png"},
|
||||
{"width": width, "height": height, "url": f"{self.base}?{width}-reloaded#home", "wait": 1200, "probe": probe(OPEN, width), "shot": f"{shot}-popover.png"},
|
||||
{"width": width, "height": height, "url": None, "wait": 0, "probe": probe(CLOSE, width), "shot": None},
|
||||
]
|
||||
return steps
|
||||
|
||||
def check(self, data: dict, browser: str) -> None:
|
||||
results = iter(data["results"])
|
||||
fitting = {"wide": False, "bar": False, "chip": True, "period": True}
|
||||
previous = ["tasks", "tokens", "cpu"]
|
||||
for width, _ in base.WIDTHS:
|
||||
chosen, opened, closed = next(results), next(results), next(results)
|
||||
chip = {"day": "2/d", "week": "2/wk", "minute": "≈ 0.02/min"}[PERIODS[width]]
|
||||
with self.subTest(browser=browser, width=width):
|
||||
self.assertNotIn("error", chosen, chosen)
|
||||
self.assertEqual(chosen["before"], {"period": BEFORE[width], "metrics": previous})
|
||||
self.assertEqual(chosen["saved"], {"period": PERIODS[width], "metrics": previous})
|
||||
self.assertEqual((chosen["after"]["period"], chosen["after"]["metrics"], chosen["after"]["values"]["tasks"]), (PERIODS[width], previous, chip))
|
||||
self.assertTrue(chosen["after"]["label"].startswith("Rates per "), chosen["after"])
|
||||
self.assertEqual((chosen["fits"], chosen["open"]), (fitting, False))
|
||||
self.assertEqual(w4.quiet(chosen["violations"]), [])
|
||||
self.assertNotIn("error", opened, opened)
|
||||
self.assertEqual((opened["kept"]["period"], opened["kept"]["metrics"], opened["kept"]["values"]["tasks"]), (PERIODS[width], previous, chip))
|
||||
self.assertTrue(opened["opened"], opened)
|
||||
self.assertEqual((opened["head"], opened["rows"], opened["current"]), (HEAD, [[metric, 7] for metric in METRICS], [PERIODS[width]] * len(METRICS)))
|
||||
self.assertEqual({period: tuple(cell) for period, cell in opened["tasks"].items()}, TASKS)
|
||||
self.assertIn("scaled from the last hour", opened["hint"])
|
||||
self.assertEqual((opened["locked"], opened["fewest"]), (True, [True, True]))
|
||||
self.assertEqual((opened["swapped"], opened["chip"]), ({"period": PERIODS[width], "metrics": CHOSEN[width]}, CHOSEN[width]))
|
||||
self.assertEqual((opened["box"], opened["fits"], opened["open"]), (True, fitting, True))
|
||||
self.assertEqual(w4.quiet(opened["violations"]), [])
|
||||
self.assertEqual({key: closed[key] for key in ("was", "escaped", "reopened", "clicked", "outside")}, dict.fromkeys(("was", "escaped", "reopened", "clicked", "outside"), True))
|
||||
self.assertEqual((closed["fits"], w4.quiet(closed["violations"])), (fitting, []))
|
||||
previous = CHOSEN[width]
|
||||
self.assertEqual(self.httpd.hub.settings.get()["rates"], {"period": PERIODS[390], "metrics": CHOSEN[390]})
|
||||
|
||||
def test_chromium_shows_the_rates_chip_keeps_its_period_over_a_reload_and_opens_and_closes_the_table_at_three_widths(self):
|
||||
data = self.drive(self.steps("chromium"), None, init=w4.GUARD)
|
||||
(OUT / "chromium-rates.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
|
||||
self.assertEqual(data["errors"], [])
|
||||
self.check(data, "chromium")
|
||||
|
||||
@unittest.skipUnless(shutil.which(os.environ.get("FIREFOX", "firefox")), "no Firefox")
|
||||
def test_firefox_shows_the_rates_chip_keeps_its_period_over_a_reload_and_opens_and_closes_the_table_at_three_widths(self):
|
||||
data = self.drive(self.steps("firefox"), None, "bidi.mjs", os.environ.get("FIREFOX", "firefox"), init=w4.GUARD)
|
||||
(OUT / "firefox-rates.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
|
||||
self.assertEqual([error for error in data["errors"] if "/favicon.ico" not in error], [])
|
||||
self.check(data, "firefox")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,225 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import hub_support
|
||||
import resources as sampler
|
||||
from hub_case import HubCase
|
||||
from kvb_orchestrator import rates
|
||||
|
||||
iso = hub_support.iso
|
||||
HUB = hub_support.HUB
|
||||
|
||||
HOUR = 3600
|
||||
DAY = 86400
|
||||
GIB = 1 << 30
|
||||
NOW = 1_800_000_000 + 1800
|
||||
EXACT_ZERO = {"value": 0.0, "approx": False, "source": None}
|
||||
|
||||
|
||||
def token(t: int, given: int, cached: int) -> dict:
|
||||
return {"t": t, "agent": "Skills", "model": "claude-opus-5-5", "input": given, "output": 0, "cache_read": cached, "cache_write": 0}
|
||||
|
||||
|
||||
def sample(t: float, cpu: float, span: int | None = None, vram: tuple[int, int] | None = None, boot: int = 50) -> dict:
|
||||
gpus = [{"name": "0000:01:00.0", "util": 10.0, "vram_used": vram[0] * GIB, "vram_total": vram[1] * GIB}, {"name": "0000:02:00.0", "asleep": True}] if vram else [{"name": "0000:01:00.0", "util": None, "vram_used": None, "vram_total": None}]
|
||||
row = {"t": t, "cpu": cpu, "ram": cpu + 1, "swap": 2.0, "disks": [{"mount": "/", "used": 50, "total": 100}, {"mount": "/boot", "used": boot, "total": 100}], "gpus": gpus}
|
||||
return row if span is None else {**row, "span": span}
|
||||
|
||||
|
||||
class TestRates(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.council = Path(tmp.name) / "council"
|
||||
self.folder = Path(tmp.name) / "kept" / "resources"
|
||||
self.council.mkdir()
|
||||
self.folder.mkdir(parents=True)
|
||||
|
||||
def register(self, tasks: dict) -> None:
|
||||
(self.council / "rasci.json").write_text(json.dumps(tasks), encoding="utf-8")
|
||||
|
||||
def lines(self, path: Path, rows: list[dict]) -> None:
|
||||
path.write_text("".join(json.dumps(row) + "\n" for row in rows), encoding="utf-8")
|
||||
|
||||
def rated(self) -> dict:
|
||||
return rates.rated(self.council, self.folder.parent, NOW)
|
||||
|
||||
def column(self, metric: str) -> dict:
|
||||
found = self.rated()
|
||||
return {period: found["rates"][period][metric] for period in rates.PERIOD_NAMES}
|
||||
|
||||
def test_done_tasks_count_in_every_window_their_end_lies_in_and_total_counts_each_done_task(self):
|
||||
ends = [NOW - 30, NOW - 1800, NOW - 2 * DAY, NOW - 20 * DAY, NOW - 200 * DAY, NOW - 800 * DAY]
|
||||
tasks = {f"work/t{index}": {"status": "done", "end": iso(end)} for index, end in enumerate(ends)}
|
||||
others = {"status": "done", "end": iso(NOW - 10)}
|
||||
self.register({**tasks, "work/legacy": {"status": "done"}, "work/open": {**others, "status": "running"}, "work/halted": {**others, "status": "stopped"}, "operator/run": others})
|
||||
counts = dict(zip(rates.PERIOD_NAMES, (1.0, 2.0, 2.0, 3.0, 4.0, 5.0, 7.0), strict=True))
|
||||
self.assertEqual(self.column("tasks"), {period: {"value": value, "approx": False, "source": None} for period, value in counts.items()})
|
||||
|
||||
def test_an_empty_window_takes_the_shortest_longer_window_with_events_scaled_down_and_marked_approx(self):
|
||||
results = [{"text": "met", "met": True, "met_at": iso(NOW - 1800)}, {"text": "open", "met": False}]
|
||||
self.register({"work/a": {"status": "running", "key_results": results}})
|
||||
commit = {"time": NOW - 5 * HOUR, "remote": "example.org/repo", "sha": "a" * 40, "values": {}}
|
||||
self.lines(self.council / "scores.jsonl", [commit, commit, {**commit, "sha": None, "time": NOW - 60}, {**commit, "sha": "b" * 40, "values": None}])
|
||||
krs = self.column("krs")
|
||||
self.assertEqual(krs["minute"], {"value": 0.01667, "approx": True, "source": "hour"})
|
||||
self.assertEqual([krs[period] for period in rates.PERIOD_NAMES[1:]], [{"value": 1.0, "approx": False, "source": None}] * 6)
|
||||
commits = self.column("commits")
|
||||
self.assertEqual(commits["minute"], {"value": 0.0006944, "approx": True, "source": "day"})
|
||||
self.assertEqual(commits["hour"], {"value": 0.04167, "approx": True, "source": "day"})
|
||||
self.assertEqual([commits[period] for period in rates.PERIOD_NAMES[2:]], [{"value": 1.0, "approx": False, "source": None}] * 5)
|
||||
|
||||
def test_key_results_met_before_met_at_existed_count_only_in_total_and_scale_from_there(self):
|
||||
self.register({"work/a": {"status": "running", "key_results": [{"text": "a", "met": True}]}})
|
||||
self.lines(self.folder / "samples.jsonl", [sample(NOW - 10 * DAY, 5.0)])
|
||||
krs = self.column("krs")
|
||||
self.assertEqual(krs["total"], {"value": 1.0, "approx": False, "source": None})
|
||||
self.assertEqual(krs["year"], {"value": 1.0, "approx": True, "source": "total"})
|
||||
self.assertEqual(krs["week"], {"value": 0.7, "approx": True, "source": "total"})
|
||||
self.assertEqual(krs["minute"], {"value": 6.944e-05, "approx": True, "source": "total"})
|
||||
|
||||
def test_tokens_share_their_hour_by_overlap_and_the_minute_always_takes_a_longer_window(self):
|
||||
hour = NOW - NOW % HOUR
|
||||
self.lines(self.folder / "tokens.jsonl", [token(hour, 1000, 800), token(hour - HOUR, 3000, 600), token(hour - 20 * DAY, 7, 0)])
|
||||
tokens = self.column("tokens")
|
||||
self.assertEqual(tokens["minute"], {"value": 60.0, "approx": True, "source": "hour"})
|
||||
self.assertEqual(tokens["hour"], {"value": 3600.0, "approx": False, "source": None})
|
||||
self.assertEqual((tokens["day"]["value"], tokens["month"]["value"], tokens["total"]["value"]), (5400.0, 5407.0, 5407.0))
|
||||
|
||||
def test_an_empty_hour_takes_the_day_for_tokens_and_so_does_the_minute(self):
|
||||
hour = NOW - NOW % HOUR
|
||||
self.lines(self.folder / "tokens.jsonl", [token(hour - 5 * HOUR, 8000, 640)])
|
||||
tokens = self.column("tokens")
|
||||
self.assertEqual(tokens["minute"], {"value": 6.0, "approx": True, "source": "day"})
|
||||
self.assertEqual(tokens["hour"], {"value": 360.0, "approx": True, "source": "day"})
|
||||
self.assertEqual(tokens["day"], {"value": 8640.0, "approx": False, "source": None})
|
||||
|
||||
def test_gauges_average_each_window_weighting_raw_rows_fifteen_minute_means_and_daily_means_by_their_span(self):
|
||||
self.lines(self.folder / "samples.jsonl", [
|
||||
sample(NOW - 500 * DAY, 0.0, span=DAY),
|
||||
sample(NOW - 200 * DAY, 90.0, span=DAY),
|
||||
sample(NOW - 10 * DAY, 50.0, span=900),
|
||||
sample(NOW - 90, 30.0, vram=(4, 8), boot=70),
|
||||
sample(NOW - 30, 10.0, vram=(2, 8), boot=90),
|
||||
])
|
||||
found = self.rated()["rates"]
|
||||
self.assertEqual([found[period]["cpu"]["value"] for period in rates.PERIOD_NAMES], [10.0, 20.0, 20.0, 20.0, 46.5, 89.5, 45.0])
|
||||
self.assertEqual([found[period]["ram"]["value"] for period in ("minute", "hour")], [11.0, 21.0])
|
||||
self.assertEqual(found["minute"]["vram"], {"value": 25.0, "used": 2 * GIB, "approx": False, "source": None})
|
||||
self.assertEqual((found["hour"]["vram"]["value"], found["hour"]["vram"]["used"], found["month"]["vram"]["value"]), (37.5, 3 * GIB, 37.5))
|
||||
self.assertEqual(found["minute"]["disk"], {"value": 90.0, "mount": "/boot", "used": 90, "total": 100, "approx": False, "source": None})
|
||||
self.assertEqual((found["hour"]["disk"]["value"], found["month"]["disk"]["value"], found["month"]["disk"]["used"]), (80.0, 53.5, 54))
|
||||
self.assertEqual(self.rated()["since"], NOW - 500 * DAY)
|
||||
|
||||
def test_a_window_without_a_sample_shows_no_gauge_and_kinds_without_any_event_count_an_exact_zero(self):
|
||||
self.lines(self.folder / "samples.jsonl", [sample(NOW - 2 * HOUR, 40.0)])
|
||||
found = self.rated()
|
||||
self.assertEqual([found["rates"][period]["cpu"]["value"] for period in rates.PERIOD_NAMES], [None, None, 40.0, 40.0, 40.0, 40.0, 40.0])
|
||||
self.assertEqual(found["rates"]["hour"]["vram"], {"value": None, "used": None, "approx": False, "source": None})
|
||||
self.assertEqual(found["rates"]["minute"]["disk"], {"value": None, "mount": None, "used": None, "total": None, "approx": False, "source": None})
|
||||
self.assertEqual({found["rates"][period][kind] == EXACT_ZERO for period in rates.PERIOD_NAMES for kind in rates.COUNTS}, {True})
|
||||
self.assertEqual((found["now"], found["since"], [entry["name"] for entry in found["periods"]]), (NOW, NOW - 2 * HOUR, list(rates.PERIOD_NAMES)))
|
||||
|
||||
def test_folded_sampler_rows_feed_the_windows_with_the_seconds_they_sampled(self):
|
||||
old = NOW - NOW % DAY - 100 * DAY
|
||||
raw = [sample(old + 3600 + index * 60, 20.0) for index in range(30)] + [sample(NOW - 9 * DAY + index * 60, 60.0) for index in range(30)] + [sample(NOW - 59, 10.0)]
|
||||
kept = sampler.retain(sorted(raw, key=lambda row: row["t"]), NOW)
|
||||
self.assertEqual([(row["t"], row["span"], row["width"]) for row in kept[:1]], [(old, 1800, DAY)])
|
||||
self.assertEqual(sum(row["span"] for row in kept[1:-1]), 1800)
|
||||
self.assertEqual({row["width"] for row in kept[1:-1]}, {900})
|
||||
self.assertNotIn("span", kept[-1])
|
||||
self.lines(self.folder / "samples.jsonl", kept)
|
||||
found = self.rated()["rates"]
|
||||
month = round((10 * 60 + 60 * 1800) / (60 + 1800), 1)
|
||||
year = round((10 * 60 + 60 * 1800 + 20 * 1800) / (60 + 1800 * 2), 1)
|
||||
self.assertEqual([found[period]["cpu"]["value"] for period in ("minute", "week", "month", "year")], [10.0, 10.0, month, year])
|
||||
|
||||
def test_a_lone_fifteen_minute_bucket_weighs_its_sampled_seconds_wherever_it_lies_in_its_day(self):
|
||||
day = NOW - NOW % DAY - 200 * DAY
|
||||
raw = [sample(day + 60 * index, 50.0) for index in range(15)] + [sample(day + DAY + 900 + 60 * index, 50.0) for index in range(15)]
|
||||
kept = sampler.retain(raw, NOW)
|
||||
self.assertEqual([(row["t"], row["span"], row["width"]) for row in kept], [(day, 900, DAY), (day + DAY, 900, DAY)])
|
||||
self.assertEqual(sampler.retain(kept, NOW), kept)
|
||||
|
||||
def test_a_fifteen_minute_mean_written_before_rows_carried_span_stands_for_its_whole_bucket(self):
|
||||
start = NOW - NOW % 900 - 20 * DAY
|
||||
legacy = {**sample(start, 30.0), "queue": {"waiting": 0.0, "active": 1.0}}
|
||||
kept = sampler.retain([legacy, sample(start + 900 + 60, 90.0)], NOW)
|
||||
self.assertEqual([(row["t"], row["span"], row["width"]) for row in kept], [(start, 900, 900), (start + 900, 60, 900)])
|
||||
|
||||
def test_the_year_weighs_every_tier_by_the_seconds_it_sampled(self):
|
||||
old, recent = NOW - NOW % DAY - 100 * DAY, NOW - NOW % DAY - 8 * DAY
|
||||
raw = [sample(old + 60 * index, 90.0) for index in range(60)] + [sample(recent + 60 * index, 10.0) for index in range(60)]
|
||||
self.lines(self.folder / "samples.jsonl", sampler.retain(raw, NOW))
|
||||
self.assertEqual(self.rated()["rates"]["year"]["cpu"]["value"], 50.0)
|
||||
|
||||
def test_a_folded_row_straddling_a_window_start_counts_by_its_share_inside(self):
|
||||
week, year = NOW - 7 * DAY, NOW - 365 * DAY
|
||||
self.lines(self.folder / "samples.jsonl", [
|
||||
{**sample(year - year % DAY, 40.0), "span": 3600, "width": DAY},
|
||||
{**sample(week - 300, 70.0, boot=80), "span": 600, "width": 900},
|
||||
sample(NOW - 30, 10.0, boot=20),
|
||||
])
|
||||
found = self.rated()["rates"]
|
||||
inside = (year - year % DAY + DAY - year) / DAY * 3600
|
||||
self.assertEqual(found["week"]["cpu"]["value"], round((400 * 70 + 60 * 10) / 460, 1))
|
||||
self.assertEqual(found["month"]["cpu"]["value"], round((600 * 70 + 60 * 10) / 660, 1))
|
||||
self.assertEqual(found["year"]["cpu"]["value"], round((inside * 40 + 600 * 70 + 60 * 10) / (inside + 660), 1))
|
||||
self.assertEqual(found["total"]["cpu"]["value"], round((3600 * 40 + 600 * 70 + 60 * 10) / 4260, 1))
|
||||
self.assertEqual(found["week"]["disk"]["value"], round((400 * 80 + 60 * 20) / 460, 1))
|
||||
|
||||
def test_token_rows_older_than_ninety_days_fold_into_one_row_per_day_agent_and_model_kept_forever(self):
|
||||
day, hour = NOW - NOW % DAY - 100 * DAY, NOW - NOW % HOUR
|
||||
rows = [token(day + HOUR, 10, 0), token(day + 5 * HOUR, 20, 5), {**token(day + 2 * HOUR, 1, 0), "model": "other"}, token(hour - HOUR, 7, 0)]
|
||||
kept = sampler.compact(rows, NOW)
|
||||
daily = [{**token(day, 30, 5), "width": DAY}, {**token(day, 1, 0), "model": "other", "width": DAY}]
|
||||
self.assertEqual(kept, [*daily, token(hour - HOUR, 7, 0)])
|
||||
self.assertEqual(sampler.compact(kept, NOW + 400 * DAY)[:2], daily)
|
||||
|
||||
def test_a_daily_token_row_counts_by_the_share_of_its_day_inside_a_window(self):
|
||||
year = NOW - 365 * DAY
|
||||
self.lines(self.folder / "tokens.jsonl", [{**token(year - year % DAY, 8640, 0), "width": DAY}])
|
||||
tokens = self.column("tokens")
|
||||
self.assertEqual(tokens["year"], {"value": rates.precise((year - year % DAY + DAY - year) / DAY * 8640), "approx": False, "source": None})
|
||||
self.assertEqual(tokens["total"], {"value": 8640.0, "approx": False, "source": None})
|
||||
|
||||
def test_token_windows_reaching_before_the_first_token_row_are_scaled_estimates_from_the_rows_there_are(self):
|
||||
hour = NOW - NOW % HOUR
|
||||
begins = hour - 10 * DAY
|
||||
self.register({"work/old": {"status": "done", "end": iso(NOW - 400 * DAY)}})
|
||||
self.lines(self.folder / "tokens.jsonl", [token(begins, 1000, 0), token(hour - HOUR, 500, 0)])
|
||||
tokens = self.column("tokens")
|
||||
covered = NOW - begins
|
||||
self.assertEqual(tokens["minute"], {"value": rates.precise(250 / 60), "approx": True, "source": "hour"})
|
||||
self.assertEqual([tokens[period] for period in ("hour", "day", "week")], [{"value": value, "approx": False, "source": None} for value in (250.0, 500.0, 500.0)])
|
||||
for period, seconds in (("month", 30 * DAY), ("year", 365 * DAY), ("total", 400 * DAY)):
|
||||
with self.subTest(period=period):
|
||||
self.assertEqual(tokens[period], {"value": rates.precise(1500 * seconds / covered), "approx": True, "source": "total"})
|
||||
|
||||
def test_counts_shorten_to_thousands_millions_billions_and_trillions(self):
|
||||
probe = "const R = require(process.argv[1]); console.log(JSON.stringify(JSON.parse(process.argv[2]).map(R.compact)));"
|
||||
values = [999, 1000, 999_949, 999_950, 14_370_239_246.104, 999_960_000_000, 2.5e12, 3e15]
|
||||
done = subprocess.run(["node", "-e", probe, str(HUB / "templates" / "resources.js"), json.dumps(values)], capture_output=True, text=True, timeout=60, check=True)
|
||||
self.assertEqual(json.loads(done.stdout), ["999", "1.0k", "999.9k", "1.0M", "14.4B", "1.0T", "2.5T", "3000.0T"])
|
||||
|
||||
|
||||
class TestHubRates(HubCase):
|
||||
def test_the_hub_serves_the_rates_of_the_register_the_journal_and_the_sampler(self):
|
||||
self.write_council({"rasci.json": {"work/a": {"status": "done", "end": iso(1_700_000_000), "key_results": [{"text": "a", "met": True, "met_at": iso(1_700_000_000)}]}}})
|
||||
(self.council / "scores.jsonl").write_text(json.dumps({"time": 1_700_000_000, "remote": "example.org/repo", "sha": "c" * 40, "values": {}}) + "\n", encoding="utf-8")
|
||||
status, headers, body = self.get("/api/rates")
|
||||
self.assertEqual((status, headers["content-type"]), (200, "application/json"), body)
|
||||
found = json.loads(body)
|
||||
self.assertEqual([entry["name"] for entry in found["periods"]], list(rates.PERIOD_NAMES))
|
||||
self.assertEqual({kind: found["rates"]["total"][kind]["value"] for kind in rates.COUNTS}, {"tasks": 1.0, "commits": 1.0, "krs": 1.0, "tokens": 0.0})
|
||||
self.assertEqual(set(found["rates"]["hour"]), set(rates.METRICS))
|
||||
self.assertEqual(found["since"], 1_700_000_000)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -194,6 +194,35 @@ class TestHubSettings(HubCase):
|
||||
(self.state / "settings.json").write_text(json.dumps(older), encoding="utf-8")
|
||||
self.assertEqual(settings.Settings(self.state / "settings.json").get()["trend"], {"mode": "off", "days": 7})
|
||||
|
||||
def test_the_rates_chip_defaults_to_tasks_tokens_and_cpu_per_hour_is_validated_stored_without_a_passkey_survives_a_reload_and_a_sheet_without_it_keeps_it(self):
|
||||
default = {"period": "hour", "metrics": ["tasks", "tokens", "cpu"]}
|
||||
self.assertEqual((settings.SETTINGS["rates"], self.snapshot()["settings"]["rates"]), (default, default))
|
||||
good = self.guard(self.host)
|
||||
wrongs = [{"period": period, "metrics": ["tasks", "cpu"]} for period in ("second", "Hour", "", None, 1)]
|
||||
wrongs += [{"period": "day", "metrics": metrics} for metrics in (["tasks"], [], ["tasks", "cpu", "ram", "swap"], ["tasks", "tasks"], ["tasks", "gpu"], [1, 2], "tasks,cpu", None)]
|
||||
wrongs += [{"period": "day"}, {"metrics": ["tasks", "cpu"]}, {**default, "extra": 1}, "day", ["day"]]
|
||||
for wrong in wrongs:
|
||||
with self.subTest(wrong=wrong):
|
||||
self.assertEqual(self.post_json("/api/settings", {**settings.SETTINGS, "rates": wrong}, good)[0], 400)
|
||||
self.assertEqual(self.snapshot()["settings"]["rates"], default)
|
||||
for chosen in ({"period": "minute", "metrics": ["commits", "krs"]}, {"period": "total", "metrics": ["disk", "vram", "swap"]}, {"period": "week", "metrics": ["ram", "tokens"]}):
|
||||
with self.subTest(chosen=chosen):
|
||||
self.assertIsNone(self.prepared({**self.snapshot()["settings"], "rates": chosen})["payload"])
|
||||
status, _, body = self.post_json("/api/settings", {**self.snapshot()["settings"], "rates": chosen, "proof": None}, good)
|
||||
self.assertEqual((status, json.loads(body)["settings"]["rates"]), (200, chosen))
|
||||
last = {"period": "week", "metrics": ["ram", "tokens"]}
|
||||
self.assertEqual(json.loads((self.state / "settings.json").read_text(encoding="utf-8"))["rates"], last)
|
||||
self.assertEqual(settings.Settings(self.state / "settings.json").get()["rates"], last)
|
||||
self.assertEqual(self.snapshot()["settings"]["rates"], last)
|
||||
sheet = {key: value for key, value in self.snapshot()["settings"].items() if key != "rates"}
|
||||
self.assertIsNone(self.prepared({**sheet, "estimation": "points"})["payload"])
|
||||
status, _, body = self.post_json("/api/settings", {**sheet, "estimation": "points", "proof": None}, good)
|
||||
self.assertEqual((status, json.loads(body)["settings"]["estimation"], json.loads(body)["settings"]["rates"]), (200, "points", last))
|
||||
self.assertEqual(settings.Settings(self.state / "settings.json").get()["rates"], last)
|
||||
older = {key: value for key, value in self.settings_payload(STREAMS).items() if key != "rates"}
|
||||
(self.state / "settings.json").write_text(json.dumps(older), encoding="utf-8")
|
||||
self.assertEqual(settings.Settings(self.state / "settings.json").get()["rates"], default)
|
||||
|
||||
def test_settings_written_before_the_radio_filter_keep_their_values(self):
|
||||
older = {key: value for key, value in self.settings_payload(STREAMS).items() if key != "speech"}
|
||||
(self.state / "settings.json").write_text(json.dumps(older), encoding="utf-8")
|
||||
|
||||
@@ -142,6 +142,30 @@ class TestRasci(unittest.TestCase):
|
||||
self.assertEqual(self._run("set", "work/a", "--status", "done").returncode, 0)
|
||||
self.assertEqual(self._out("show"), ["no tasks"])
|
||||
|
||||
def test_a_key_result_set_met_records_met_at_once_and_open_drops_it(self):
|
||||
self._run("kr", "work/a", "add", "tests green")
|
||||
self.assertEqual(self._first_kr("work/a"), {"text": "tests green", "met": False})
|
||||
before = datetime.datetime.now(datetime.UTC).replace(microsecond=0)
|
||||
self._out("kr", "work/a", "met", "1")
|
||||
met = self._first_kr("work/a")
|
||||
moment = datetime.datetime.fromisoformat(met["met_at"])
|
||||
self.assertEqual((met["met"], moment.utcoffset()), (True, datetime.timedelta(0)))
|
||||
self.assertLessEqual(before, moment)
|
||||
self.assertLessEqual(moment, datetime.datetime.now(datetime.UTC))
|
||||
self._out("kr", "work/a", "met", "1")
|
||||
self.assertEqual(self._first_kr("work/a"), met)
|
||||
self._out("kr", "work/a", "open", "1")
|
||||
self.assertEqual(self._first_kr("work/a"), {"text": "tests green", "met": False})
|
||||
|
||||
def test_a_key_result_met_before_met_at_existed_stays_unstamped_when_met_again(self):
|
||||
self.root.mkdir()
|
||||
(self.root / "rasci.json").write_text(json.dumps({"work/a": {"roles": {}, "key_results": [{"text": "old", "met": True}]}}), encoding="utf-8")
|
||||
self._out("kr", "work/a", "met", "1")
|
||||
self.assertEqual(self._first_kr("work/a"), {"text": "old", "met": True})
|
||||
|
||||
def _first_kr(self, task: str) -> dict:
|
||||
return json.loads((self.root / "rasci.json").read_text(encoding="utf-8"))[task]["key_results"][0]
|
||||
|
||||
def test_status_sets_start_and_end(self):
|
||||
self._run("set", "work/a", "--status", "running")
|
||||
self._run("set", "work/a", "--status", "done", "--description", "long text")
|
||||
|
||||
@@ -178,7 +178,7 @@ class TestGpus(unittest.TestCase):
|
||||
unknown = {"name": AWAKE, "util": None, "vram_used": None, "vram_total": None}
|
||||
self.assertEqual(sampler.gpus(host_sys(self.root), lambda argv: None), [unknown, {"name": ASLEEP, "asleep": True}])
|
||||
self.assertEqual(sampler.gpus(self.root, lambda argv: NVIDIA_SMI.splitlines()[1]), [unknown, {"name": ASLEEP, "asleep": True}])
|
||||
self.assertEqual(sampler.mean_row(NOW, [row(NOW + 60, 1.0, 1, 0, unknown), row(NOW + 120, 1.0, 1, 0, unknown)])["gpus"], [unknown])
|
||||
self.assertEqual(sampler.mean_row(NOW, sampler.BUCKET, [row(NOW + 60, 1.0, 1, 0, unknown), row(NOW + 120, 1.0, 1, 0, unknown)])["gpus"], [unknown])
|
||||
|
||||
def test_no_nvidia_gpu_means_no_gpu_row_and_no_call(self):
|
||||
device(self.root, "0000:00:02.0", "0x8086", "0x030000", "active")
|
||||
@@ -188,12 +188,13 @@ class TestGpus(unittest.TestCase):
|
||||
|
||||
|
||||
class TestRetention(unittest.TestCase):
|
||||
def test_rows_older_than_seven_days_fold_into_fifteen_minute_means_and_means_past_ninety_days_drop(self):
|
||||
def test_rows_older_than_seven_days_fold_into_fifteen_minute_means_and_rows_past_ninety_days_into_their_day(self):
|
||||
bucket = NOW - 8 * DAY
|
||||
day = NOW - 91 * DAY - NOW % DAY
|
||||
asleep, awake = {"name": AWAKE, "asleep": True}, {"name": AWAKE, "util": 50.0, "vram_used": 300, "vram_total": 1000}
|
||||
mean = {**row(NOW - 30 * DAY, 12.0, 100, 0, asleep)}
|
||||
rows = [
|
||||
row(NOW - 91 * DAY, 5.0, 1, 0, asleep),
|
||||
row(NOW - 91 * DAY + 60, 5.0, 1, 0, asleep),
|
||||
mean,
|
||||
row(bucket, 10.0, 100, 0, asleep),
|
||||
row(bucket + 60, 20.0, 200, 1, asleep),
|
||||
@@ -202,16 +203,37 @@ class TestRetention(unittest.TestCase):
|
||||
row(NOW - 60, 70.0, 700, 3, awake),
|
||||
]
|
||||
kept = sampler.retain(rows, NOW)
|
||||
self.assertEqual([entry["t"] for entry in kept], [NOW - 30 * DAY, bucket, bucket + 900, NOW - 60])
|
||||
self.assertIs(kept[0], mean)
|
||||
self.assertEqual(kept[1], {
|
||||
self.assertEqual([entry["t"] for entry in kept], [day, NOW - 30 * DAY, bucket, bucket + 900, NOW - 60])
|
||||
self.assertEqual(kept[0], {**row(day, 5.0, 1, 0, asleep), "queue": {"waiting": 0.0, "active": 1.0}, "span": 60, "width": DAY})
|
||||
self.assertEqual(kept[1], {**mean, "span": sampler.BUCKET, "width": sampler.BUCKET})
|
||||
self.assertEqual(kept[2], {
|
||||
"t": bucket, "disks": [{"mount": "/", "used": 200, "total": 1000}, {"mount": "/mnt", "used": 10, "total": 20}], "cpu": 30.0, "ram": 15.0, "swap": 0.0, "load": 3.0,
|
||||
"gpus": [{"name": AWAKE, "util": 50.0, "vram_used": 300, "vram_total": 1000}], "queue": {"waiting": 1.0, "active": 1.0},
|
||||
"gpus": [{"name": AWAKE, "util": 50.0, "vram_used": 300, "vram_total": 1000}], "queue": {"waiting": 1.0, "active": 1.0}, "span": 180, "width": sampler.BUCKET,
|
||||
})
|
||||
self.assertEqual((kept[2]["cpu"], kept[2]["disks"]), (40.0, [{"mount": "/", "used": 400, "total": 1000}]))
|
||||
self.assertEqual(kept[3], rows[-1])
|
||||
self.assertEqual((kept[3]["cpu"], kept[3]["disks"], kept[3]["span"], kept[3]["width"]), (40.0, [{"mount": "/", "used": 400, "total": 1000}], 60, sampler.BUCKET))
|
||||
self.assertEqual(kept[4], rows[-1])
|
||||
self.assertEqual(sampler.retain(kept, NOW), kept)
|
||||
|
||||
def test_days_that_ended_more_than_ninety_days_ago_fold_into_one_mean_each_kept_forever(self):
|
||||
asleep = {"name": AWAKE, "asleep": True}
|
||||
day = NOW - NOW % DAY - 100 * DAY
|
||||
edge = NOW - NOW % DAY - 90 * DAY
|
||||
ancient = {**row(day - 1000 * DAY, 7.0, 7, 0, asleep), "span": 3600, "width": DAY}
|
||||
means = [{**row(day + index * sampler.BUCKET, cpu, 100, 0, asleep), "span": sampler.BUCKET, "width": sampler.BUCKET} for index, cpu in enumerate((10.0, 20.0, 60.0))]
|
||||
late = {**row(edge + sampler.BUCKET, 40.0, 400, 0, asleep), "span": sampler.BUCKET, "width": sampler.BUCKET}
|
||||
recent = row(NOW - 60, 70.0, 700, 3, asleep)
|
||||
kept = sampler.retain([ancient, *means, late, recent], NOW)
|
||||
self.assertEqual([entry["t"] for entry in kept], [day - 1000 * DAY, day, edge + sampler.BUCKET, NOW - 60])
|
||||
self.assertEqual(kept[0], ancient)
|
||||
self.assertEqual(kept[1], {**row(day, 30.0, 100, 0, asleep), "load": 3.0, "queue": {"waiting": 0.0, "active": 1.0}, "span": 3 * sampler.BUCKET, "width": DAY})
|
||||
self.assertEqual(kept[2:], [late, recent])
|
||||
self.assertEqual(sampler.retain(kept, NOW), kept)
|
||||
later = sampler.retain(kept, NOW + 2 * DAY)
|
||||
self.assertEqual([entry["t"] for entry in later], [day - 1000 * DAY, day, edge, NOW - 60])
|
||||
self.assertEqual(later[:2], kept[:2])
|
||||
self.assertEqual((later[2]["cpu"], later[2]["span"], later[2]["width"]), (40.0, sampler.BUCKET, DAY))
|
||||
self.assertEqual(sampler.retain(later, NOW + 400 * DAY)[:3], later[:3])
|
||||
|
||||
def test_a_bucket_still_reaching_into_the_last_seven_days_stays_raw_and_asleep_buckets_stay_asleep(self):
|
||||
now = NOW + 300
|
||||
asleep = {"name": ASLEEP, "asleep": True}
|
||||
|
||||
@@ -170,16 +170,19 @@ class TestTokens(TokensCase):
|
||||
self.assertEqual(self.count(now=NOW + 180), [])
|
||||
self.assertEqual(self.offsets(), stored)
|
||||
|
||||
def test_the_first_minute_of_an_hour_merges_rows_and_drops_those_older_than_ninety_days(self):
|
||||
def test_the_first_minute_of_an_hour_merges_rows_and_folds_days_past_ninety_days_into_one_row_kept_forever(self):
|
||||
hour = NOW
|
||||
stale = {"t": hour - 91 * 86400, "agent": "Old", "model": OPUS, "input": 1, "output": 1, "cache_read": 1, "cache_write": 1}
|
||||
stale = {"t": hour - 92 * 86400, "agent": "Old", "model": OPUS, "input": 1, "output": 1, "cache_read": 1, "cache_write": 1}
|
||||
parts = [{"t": hour - HOUR, "agent": "Skills", "model": OPUS, "input": value, "output": 0, "cache_read": 0, "cache_write": 0} for value in (1, 2, 3)]
|
||||
(self.state / "tokens.jsonl").write_text(lines([stale, *parts]), encoding="utf-8")
|
||||
self.transcript.write_text(lines([assistant(NOW + 10, SESSION, OPUS, "msg_1", (4, 0, 0, 0))]), encoding="utf-8")
|
||||
self.count(now=hour + 1800)
|
||||
self.assertEqual(len(self.written()), 5)
|
||||
self.count(now=hour + 30)
|
||||
self.assertEqual(self.written(), [{**parts[0], "input": 6}, {"t": hour, "agent": "Skills", "model": OPUS, "input": 4, "output": 0, "cache_read": 0, "cache_write": 0}])
|
||||
daily = {**stale, "t": stale["t"] - stale["t"] % 86400, "width": 86400}
|
||||
self.assertEqual(self.written(), [daily, {**parts[0], "input": 6}, {"t": hour, "agent": "Skills", "model": OPUS, "input": 4, "output": 0, "cache_read": 0, "cache_write": 0}])
|
||||
self.count(now=hour + 400 * 86400 + 30)
|
||||
self.assertEqual(self.written()[0], daily)
|
||||
|
||||
|
||||
class TestNoLanguageModel(unittest.TestCase):
|
||||
|
||||
@@ -214,6 +214,27 @@ class TestScoreJournal(ScoreCase):
|
||||
self.score(SCORE, *flags({}), "--table", cwd=repo)
|
||||
self.assertEqual(len(self.rows()), 2)
|
||||
|
||||
def test_the_commit_table_of_a_commit_records_its_sha_and_commit_time_once_per_remote_and_sha(self):
|
||||
repo = self.repo()
|
||||
env = git_env(self.tmp)
|
||||
sha = subprocess.run(["git", "rev-parse", "HEAD"], cwd=repo, env=env, capture_output=True, text=True, check=True).stdout.strip()
|
||||
stamp = int(subprocess.run(["git", "show", "-s", "--format=%ct", sha], cwd=repo, env=env, capture_output=True, text=True, check=True).stdout)
|
||||
self.score(SCORE, *flags({}), "--message-block", cwd=repo)
|
||||
for _ in range(2):
|
||||
done = self.score(SCORE, *flags({}), "--commit-table", str(repo), "--commit", sha[:7], cwd=self.tmp)
|
||||
self.assertEqual((done.returncode, done.stderr), (0, ""))
|
||||
rendered, committed = self.rows()
|
||||
self.assertEqual(committed, {**rendered, "time": stamp, "sha": sha})
|
||||
self.score(SCORE, *flags({"fix": "80"}), "--commit-table", str(repo), cwd=self.tmp)
|
||||
self.assertEqual(len(self.rows()), 2)
|
||||
self.council.joinpath(JOURNAL_NAME).unlink()
|
||||
self.council.rmdir()
|
||||
self.council.write_text("not a directory", encoding="utf-8")
|
||||
failed = self.score(SCORE, *flags({"critical": "1"}), "--commit-table", str(repo), "--commit", sha, cwd=self.tmp)
|
||||
self.assertEqual(failed.returncode, 4)
|
||||
self.assertEqual(len(failed.stderr.splitlines()), 1, failed.stderr)
|
||||
self.assertTrue(failed.stderr.startswith(f"score: journal {self.council / JOURNAL_NAME}: "), failed.stderr)
|
||||
|
||||
def test_a_failed_append_prints_one_stderr_line_and_keeps_the_exit_code(self):
|
||||
self.council.write_text("not a directory", encoding="utf-8")
|
||||
for values, code in (({}, 0), ({"efficiency": "120"}, 3), ({"break": "9"}, 1), ({"critical": "1"}, 4)):
|
||||
|
||||
Reference in new issue
Block a user