feat(kvb-check-ansible): rank where an Ansible run spends its time
New skill with ansible_tasks.py: gaps, durations, kinds, kind and task rank a run's log by wall-clock gaps or profile_tasks lines; profile-report merges cProfile dumps of the controller and its forked workers (sitecustomize hook); bench-carrier and the bench playbook measure per-task overhead against variable size. kvb-code-tune hands its Ansible timing to it. 🎯 Fix: 92 % 💥 Break: 2 % 💸 Cost: new ⏱️ Time: new 🛡️ Security: 100 % 🏆 Score: 90 🚦 Gate: cost new, time new: cleared by the operator's commit word for kvb-check-ansible (2026-11-07) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8b0c287ada
commit
05cc891998
17 files changed
+1212
No files matched your search
@@ -68,6 +68,7 @@ flowchart LR
|
|||||||
check --> check_gate[kvb-check-gate]
|
check --> check_gate[kvb-check-gate]
|
||||||
check --> check_efficiency[kvb-check-efficiency]
|
check --> check_efficiency[kvb-check-efficiency]
|
||||||
check --> check_speed[kvb-check-speed]
|
check --> check_speed[kvb-check-speed]
|
||||||
|
check --> check_ansible[kvb-check-ansible]
|
||||||
check --> check_score[kvb-check-score]
|
check --> check_score[kvb-check-score]
|
||||||
org --> org_organize[kvb-org-organize]
|
org --> org_organize[kvb-org-organize]
|
||||||
org --> org_split[kvb-org-split]
|
org --> org_split[kvb-org-split]
|
||||||
@@ -173,6 +174,7 @@ flowchart LR
|
|||||||
gate[kvb-check-gate]
|
gate[kvb-check-gate]
|
||||||
efficiency[kvb-check-efficiency]
|
efficiency[kvb-check-efficiency]
|
||||||
speed[kvb-check-speed]
|
speed[kvb-check-speed]
|
||||||
|
check_ansible[kvb-check-ansible]
|
||||||
score[kvb-check-score]
|
score[kvb-check-score]
|
||||||
end
|
end
|
||||||
subgraph cat_org [org]
|
subgraph cat_org [org]
|
||||||
@@ -306,6 +308,8 @@ flowchart LR
|
|||||||
tune --> funnel
|
tune --> funnel
|
||||||
tune --> merge_fix
|
tune --> merge_fix
|
||||||
tune --> dialectic
|
tune --> dialectic
|
||||||
|
tune --> check_ansible
|
||||||
|
check_ansible -. heavy commands .-> queue
|
||||||
|
|
||||||
dialectic --> active_listening
|
dialectic --> active_listening
|
||||||
criticize --> dialectic
|
criticize --> dialectic
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
---
|
||||||
|
name: kvb-check-ansible
|
||||||
|
description: >
|
||||||
|
Rank where an Ansible run spends its time from its log, profile the
|
||||||
|
controller, bench per-task overhead. Trigger on a slow playbook or deploy.
|
||||||
|
---
|
||||||
|
|
||||||
|
Where an Ansible run spends its time: its log, a controller profile, a bench.
|
||||||
|
Report the script's lines with their log and range, never a guess. The
|
||||||
|
script and the bench are light runs (`watch.py admit --kind test-nostack` of
|
||||||
|
`kvb-org-organize`); a deploy to profile is heavy and goes through the
|
||||||
|
`kvb-org-queue` skill.
|
||||||
|
|
||||||
|
## 1. Pick the log
|
||||||
|
|
||||||
|
- The newest log of the current code; an older one may time a task since
|
||||||
|
replaced.
|
||||||
|
- A log whose lines start with an ISO timestamp (GitHub Actions) times every
|
||||||
|
task by wall clock. A local run needs
|
||||||
|
`ANSIBLE_CALLBACKS_ENABLED=ansible.posix.profile_tasks`, streamed to a file.
|
||||||
|
- One deploy among several: bound it with `--from` and `--to`, line numbers
|
||||||
|
from `grep -n 'PLAY \['`.
|
||||||
|
|
||||||
|
## 2. Rank
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 ${CLAUDE_SKILL_DIR}/scripts/ansible_tasks.py gaps --log <log> --top 20
|
||||||
|
```
|
||||||
|
|
||||||
|
| Subcommand | Prints |
|
||||||
|
| --- | --- |
|
||||||
|
| `gaps` | wall seconds and count per task name, then `TOTAL` |
|
||||||
|
| `durations` | the same from `profile_tasks` `(h:mm:ss.sss)` lines |
|
||||||
|
| `kinds` | per kind (ran, skipped, included, other): count, sum, mean, buckets `<0.25` to `>=2` s |
|
||||||
|
| `kind --kind <kind>` | per task name, that kind only |
|
||||||
|
| `task --needle <text>` | every occurrence: start line, seconds, kind |
|
||||||
|
|
||||||
|
`kinds`, `kind` and `task` read wall gaps when a header carries a timestamp,
|
||||||
|
else `profile_tasks` lines; `--source wall|profile` forces one. A task runs
|
||||||
|
from its header to the next header or `PLAY RECAP`. `--json` prints JSON;
|
||||||
|
exit 2 means nothing measurable in the range.
|
||||||
|
|
||||||
|
Rank by sum, a name's cost times its count: 400 skips of 0.3 s outweigh one
|
||||||
|
60 s download.
|
||||||
|
|
||||||
|
## 3. Profile the controller
|
||||||
|
|
||||||
|
A skipped task does no work on any host; its time is controller overhead
|
||||||
|
(templating, variable merging, one worker fork per task). Profile only when
|
||||||
|
`kinds` shows it: a `skipped` mean of 0.1 s or more, or `skipped` and
|
||||||
|
`included` together holding 10 % of the sum or more.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ANSIBLE_PROFILE_DIR=<new dir> PYTHONPATH=${CLAUDE_SKILL_DIR}/scripts/profile ansible-playbook <the run's arguments>
|
||||||
|
python3 ${CLAUDE_SKILL_DIR}/scripts/ansible_tasks.py profile-report --dir <new dir> --top 30
|
||||||
|
```
|
||||||
|
|
||||||
|
- `sitecustomize.py` profiles only a process named `ansible-playbook`: the
|
||||||
|
controller into `main-<pid>.prof`, each forked worker into
|
||||||
|
`worker-<pid>.prof`; without `ANSIBLE_PROFILE_DIR` into
|
||||||
|
`$TMPDIR/ansible-profile`.
|
||||||
|
- The report merges every file of the directory: a new one per run.
|
||||||
|
- `PYTHONPATH` must reach the controller's interpreter (virtualenv,
|
||||||
|
container) and shadows a distribution's own `sitecustomize`.
|
||||||
|
- A profiled run is slower: compare profiles with profiles, never its wall
|
||||||
|
clock with an unprofiled run.
|
||||||
|
|
||||||
|
## 4. Bench the per-task overhead
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 ${CLAUDE_SKILL_DIR}/scripts/ansible_tasks.py bench-carrier --kb <size> --out <dir>/carrier.json
|
||||||
|
ANSIBLE_CALLBACKS_ENABLED=ansible.posix.profile_tasks ansible-playbook -i ${CLAUDE_SKILL_DIR}/bench/hosts.yml ${CLAUDE_SKILL_DIR}/bench/bench.yml -e bench_count=200 -e bench_carrier_file=<dir>/carrier.json > <dir>/bench-<size>.log 2>&1
|
||||||
|
python3 ${CLAUDE_SKILL_DIR}/scripts/ansible_tasks.py kinds --log <dir>/bench-<size>.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Each item includes one skipped and one run `debug` task on localhost. Run
|
||||||
|
once without `bench_carrier_file` and once per `<size>` in KiB, up to the
|
||||||
|
run's biggest variable; the change of the `skipped` mean is that fact's cost
|
||||||
|
per task. Without `ansible.posix`, divide the wall-time difference of
|
||||||
|
`bench_count=0` and `bench_count=200` by 400.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
Per finding: subcommand, log and range, its line as printed, and the saving
|
||||||
|
ceiling, count times the measured cost per task. Hand the ranking to the
|
||||||
|
`kvb-code-tune` loop.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
- name: Bench the controller cost per task
|
||||||
|
hosts: bench
|
||||||
|
gather_facts: false
|
||||||
|
tasks:
|
||||||
|
- name: Park the synthetic fact
|
||||||
|
ansible.builtin.set_fact:
|
||||||
|
bench_carrier: "{{ lookup('ansible.builtin.file', bench_carrier_file) | from_json }}"
|
||||||
|
when: bench_carrier_file | default('') | length > 0
|
||||||
|
|
||||||
|
- name: Include one skipped and one run task per item
|
||||||
|
ansible.builtin.include_tasks: one.yml
|
||||||
|
loop: "{{ range(0, bench_count | int) | list }}"
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
all:
|
||||||
|
hosts:
|
||||||
|
bench:
|
||||||
|
ansible_connection: local
|
||||||
|
ansible_python_interpreter: "{{ ansible_playbook_python }}"
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
- name: Skipped
|
||||||
|
ansible.builtin.debug:
|
||||||
|
msg: never
|
||||||
|
when: false
|
||||||
|
|
||||||
|
- name: Ran
|
||||||
|
ansible.builtin.debug:
|
||||||
|
msg: once
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""Write the synthetic JSON fact the bench playbook parks on its host.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
size: exact size of the JSON text in bytes.
|
||||||
|
kb: exact size of the written file in KiB.
|
||||||
|
out: file the fact goes to.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
A dict of nested service entries, many small objects as real inventories carry them,
|
||||||
|
filled up to the exact size by one pad string. write returns a (report, lines) pair.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ENTRY = {
|
||||||
|
"services": {
|
||||||
|
f"s{service}": {
|
||||||
|
"image": "x" * 20,
|
||||||
|
"version": "1.2.3",
|
||||||
|
"env": {f"K{key}": "v" * 10 for key in range(10)},
|
||||||
|
}
|
||||||
|
for service in range(5)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
PAD = "pad"
|
||||||
|
|
||||||
|
|
||||||
|
def carrier(size: int) -> dict:
|
||||||
|
value = len(json.dumps(ENTRY))
|
||||||
|
room = len(f', "{PAD}": ""')
|
||||||
|
fact: dict[str, object] = {}
|
||||||
|
used = len("{}")
|
||||||
|
while True:
|
||||||
|
key = f"entry-{len(fact)}"
|
||||||
|
grow = len(", ") * bool(fact) + len(json.dumps(key)) + len(": ") + value
|
||||||
|
if used + grow + room > size:
|
||||||
|
break
|
||||||
|
fact[key] = ENTRY
|
||||||
|
used += grow
|
||||||
|
fact[PAD] = "p" * (size - used - len(", ") * bool(fact) - len(f'"{PAD}": ""'))
|
||||||
|
return fact
|
||||||
|
|
||||||
|
|
||||||
|
def write(kb: int, out: Path) -> tuple[dict, list[str]]:
|
||||||
|
fact = carrier(kb * 1024)
|
||||||
|
text = json.dumps(fact)
|
||||||
|
out.write_text(text, encoding="ascii")
|
||||||
|
report = {"out": str(out), "bytes": len(text), "entries": len(fact) - 1}
|
||||||
|
return report, [f"{report['out']}\t{report['bytes']} bytes\t{report['entries']} entries"]
|
||||||
@@ -0,0 +1,170 @@
|
|||||||
|
"""Parse Ansible output into timed task occurrences.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
path: the Ansible output to read.
|
||||||
|
first, last: first and last line to read, 1-based and inclusive; None reads to that end.
|
||||||
|
source: wall (ISO timestamps) or profile (profile_tasks); None picks wall when a header
|
||||||
|
in range carries a timestamp, else profile.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
A task lasts from its TASK or RUNNING HANDLER header to the next header or PLAY RECAP;
|
||||||
|
profile_tasks prints that duration under the next header. Its kind is ran on an ok,
|
||||||
|
changed, failed or fatal result line, else skipped on a skipping line, else included
|
||||||
|
on an included line, else other. A stamp without zone counts as UTC. Unmeasurable
|
||||||
|
names what the range lacks.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import calendar
|
||||||
|
import re
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
HEADER = re.compile(r"(?:TASK|RUNNING HANDLER) \[(?P<name>.*)\][\s*]*$")
|
||||||
|
RECAP = re.compile(r"PLAY RECAP \*")
|
||||||
|
STAMP = re.compile(
|
||||||
|
r"^(?P<day>\d{4}-\d{2}-\d{2})[T ](?P<hour>\d{2}):(?P<minute>\d{2}):(?P<second>\d{2})"
|
||||||
|
r"(?:[.,](?P<fraction>\d+))?(?P<zone>Z|[+-]\d{2}:?\d{2})? ?"
|
||||||
|
)
|
||||||
|
ELAPSED = re.compile(r"\((?P<hours>\d+):(?P<minutes>\d+):(?P<seconds>\d+(?:\.\d+)?)\)")
|
||||||
|
ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
|
||||||
|
RESULTS = (
|
||||||
|
("ran", re.compile(r"^\s*(?:ok|changed|failed|fatal): \[")),
|
||||||
|
("skipped", re.compile(r"^\s*skipping: \[")),
|
||||||
|
("included", re.compile(r"^\s*included: ")),
|
||||||
|
)
|
||||||
|
OTHER = "other"
|
||||||
|
KINDS = (*(kind for kind, _ in RESULTS), OTHER)
|
||||||
|
SOURCES = ("wall", "profile")
|
||||||
|
|
||||||
|
Rows = Iterator[tuple[int, float | None, str]]
|
||||||
|
|
||||||
|
|
||||||
|
class Unmeasurable(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class NoStamps(Unmeasurable):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class Task:
|
||||||
|
def __init__(self, name: str, line: int, start: float | None) -> None:
|
||||||
|
self.name = name
|
||||||
|
self.line = line
|
||||||
|
self.start = start
|
||||||
|
self.seconds = 0.0
|
||||||
|
self.seen: set[str] = set()
|
||||||
|
|
||||||
|
def note(self, content: str) -> None:
|
||||||
|
self.seen.update(kind for kind, pattern in RESULTS if pattern.search(content))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def kind(self) -> str:
|
||||||
|
return next((kind for kind, _ in RESULTS if kind in self.seen), OTHER)
|
||||||
|
|
||||||
|
|
||||||
|
def offset(zone: str | None) -> int:
|
||||||
|
if zone is None or zone == "Z":
|
||||||
|
return 0
|
||||||
|
digits = zone[1:].replace(":", "")
|
||||||
|
seconds = int(digits[:2]) * 3600 + int(digits[2:]) * 60
|
||||||
|
return -seconds if zone.startswith("-") else seconds
|
||||||
|
|
||||||
|
|
||||||
|
def epoch(match: re.Match[str]) -> float:
|
||||||
|
year, month, day = (int(part) for part in match["day"].split("-"))
|
||||||
|
whole = calendar.timegm((year, month, day, int(match["hour"]), int(match["minute"]), int(match["second"])))
|
||||||
|
fraction = float(f"0.{match['fraction']}") if match["fraction"] else 0.0
|
||||||
|
return whole + fraction - offset(match["zone"])
|
||||||
|
|
||||||
|
|
||||||
|
def split_stamp(text: str) -> tuple[float | None, str]:
|
||||||
|
match = STAMP.match(text)
|
||||||
|
if match is None:
|
||||||
|
return None, text
|
||||||
|
return epoch(match), text[match.end():]
|
||||||
|
|
||||||
|
|
||||||
|
def read(path: Path, first: int | None, last: int | None) -> Rows:
|
||||||
|
with path.open(encoding="utf-8-sig", errors="replace") as handle:
|
||||||
|
for number, raw in enumerate(handle, 1):
|
||||||
|
if first is not None and number < first:
|
||||||
|
continue
|
||||||
|
if last is not None and number > last:
|
||||||
|
return
|
||||||
|
stamp, content = split_stamp(ANSI.sub("", raw.rstrip("\r\n")))
|
||||||
|
yield number, stamp, content
|
||||||
|
|
||||||
|
|
||||||
|
def boundary(content: str) -> tuple[bool, str | None]:
|
||||||
|
header = HEADER.search(content)
|
||||||
|
if header is not None:
|
||||||
|
return True, header["name"]
|
||||||
|
return RECAP.search(content) is not None, None
|
||||||
|
|
||||||
|
|
||||||
|
def wall(rows: Rows) -> list[Task]:
|
||||||
|
done: list[Task] = []
|
||||||
|
current: Task | None = None
|
||||||
|
stamped = False
|
||||||
|
for number, stamp, content in rows:
|
||||||
|
found, name = boundary(content)
|
||||||
|
if not found:
|
||||||
|
if current is not None:
|
||||||
|
current.note(content)
|
||||||
|
continue
|
||||||
|
if stamp is None:
|
||||||
|
continue
|
||||||
|
stamped = True
|
||||||
|
if current is not None:
|
||||||
|
current.seconds = stamp - current.start
|
||||||
|
done.append(current)
|
||||||
|
current = None if name is None else Task(name, number, stamp)
|
||||||
|
if not stamped:
|
||||||
|
raise NoStamps("no TASK, RUNNING HANDLER or PLAY RECAP header with an ISO timestamp in range")
|
||||||
|
if not done:
|
||||||
|
raise Unmeasurable("no task closed by a later timestamped header in range")
|
||||||
|
return done
|
||||||
|
|
||||||
|
|
||||||
|
def elapsed(match: re.Match[str]) -> float:
|
||||||
|
return int(match["hours"]) * 3600 + int(match["minutes"]) * 60 + float(match["seconds"])
|
||||||
|
|
||||||
|
|
||||||
|
def profile(rows: Rows) -> list[Task]:
|
||||||
|
done: list[Task] = []
|
||||||
|
current: Task | None = None
|
||||||
|
waiting: Task | None = None
|
||||||
|
for number, _, content in rows:
|
||||||
|
found, name = boundary(content)
|
||||||
|
if found:
|
||||||
|
if current is not None:
|
||||||
|
waiting = current
|
||||||
|
current = None if name is None else Task(name, number, None)
|
||||||
|
continue
|
||||||
|
duration = ELAPSED.search(content)
|
||||||
|
if duration is not None and waiting is not None:
|
||||||
|
waiting.seconds = elapsed(duration)
|
||||||
|
done.append(waiting)
|
||||||
|
waiting = None
|
||||||
|
elif current is not None:
|
||||||
|
current.note(content)
|
||||||
|
if not done:
|
||||||
|
raise Unmeasurable(
|
||||||
|
"no profile_tasks (h:mm:ss.sss) duration under a header in range; "
|
||||||
|
"enable the ansible.posix.profile_tasks callback, or use gaps on a timestamped log"
|
||||||
|
)
|
||||||
|
return done
|
||||||
|
|
||||||
|
|
||||||
|
def tasks(path: Path, first: int | None, last: int | None, source: str | None) -> tuple[str, list[Task]]:
|
||||||
|
if source == "profile":
|
||||||
|
return source, profile(read(path, first, last))
|
||||||
|
if source == "wall":
|
||||||
|
return source, wall(read(path, first, last))
|
||||||
|
try:
|
||||||
|
return "wall", wall(read(path, first, last))
|
||||||
|
except NoStamps:
|
||||||
|
return "profile", profile(read(path, first, last))
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
"""Merge the main and worker cProfile files of an ansible-playbook run.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
directory: profile directory; the default of profile/sitecustomize.py when None.
|
||||||
|
top: functions per ranking.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
A (report, lines) pair: per role (main, worker) the profile count, own seconds and the
|
||||||
|
top functions by cumulative and by own time. Unmeasurable when the directory holds no
|
||||||
|
profile or an unreadable one.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import pstats
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import ansible_log
|
||||||
|
|
||||||
|
PROFILER = Path(__file__).resolve().parent / "profile" / "sitecustomize.py"
|
||||||
|
ROLES = ("main", "worker")
|
||||||
|
TOTAL_TIME = 2
|
||||||
|
CUMULATIVE_TIME = 3
|
||||||
|
|
||||||
|
|
||||||
|
def default_dir() -> Path:
|
||||||
|
spec = importlib.util.spec_from_file_location("ansible_profile_site", PROFILER)
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return Path(module.profile_dir())
|
||||||
|
|
||||||
|
|
||||||
|
def hottest(stats: pstats.Stats, field: int, top: int) -> list[dict]:
|
||||||
|
rows = sorted(stats.stats.items(), key=lambda item: -item[1][field])[:top]
|
||||||
|
return [
|
||||||
|
{"function": pstats.func_std_string(func), "calls": calls, "tottime": round(own, 6), "cumtime": round(cumulative, 6)}
|
||||||
|
for func, (_, calls, own, cumulative, _) in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def merged(directory: Path, top: int) -> dict[str, dict]:
|
||||||
|
roles: dict[str, dict] = {}
|
||||||
|
for role in ROLES:
|
||||||
|
files = sorted(directory.glob(f"{role}-*.prof"))
|
||||||
|
if not files:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
stats = pstats.Stats(*(str(file) for file in files))
|
||||||
|
except (EOFError, TypeError, ValueError) as error:
|
||||||
|
raise ansible_log.Unmeasurable(f"unreadable profile in {directory}: {error}") from None
|
||||||
|
roles[role] = {
|
||||||
|
"profiles": len(files),
|
||||||
|
"seconds": round(stats.total_tt, 3),
|
||||||
|
"cumulative": hottest(stats, CUMULATIVE_TIME, top),
|
||||||
|
"tottime": hottest(stats, TOTAL_TIME, top),
|
||||||
|
}
|
||||||
|
if not roles:
|
||||||
|
raise ansible_log.Unmeasurable(f"no main-*.prof or worker-*.prof in {directory}")
|
||||||
|
return roles
|
||||||
|
|
||||||
|
|
||||||
|
def report(directory: Path | None, top: int) -> tuple[dict, list[str]]:
|
||||||
|
chosen = default_dir() if directory is None else directory
|
||||||
|
roles = merged(chosen, top)
|
||||||
|
lines: list[str] = []
|
||||||
|
for role, found in roles.items():
|
||||||
|
lines.append(f"== {role}: {found['profiles']} profiles, {found['seconds']:.3f} s ==")
|
||||||
|
for order in ("cumulative", "tottime"):
|
||||||
|
lines.append(f"-- {order}: cumtime, tottime, calls, function --")
|
||||||
|
lines += [
|
||||||
|
f"{row['cumtime']:.3f}\t{row['tottime']:.3f}\t{row['calls']}\t{row['function']}" for row in found[order]
|
||||||
|
]
|
||||||
|
return {"dir": str(chosen), **roles}, lines
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""Rank timed task occurrences into the reports of durations, gaps, kinds, kind and task.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
source: the source the occurrences came from, echoed in the JSON report.
|
||||||
|
found: the task occurrences of ansible_log.
|
||||||
|
top: rows per table; every row when None.
|
||||||
|
kind: ran, skipped, included or other.
|
||||||
|
needle: substring of the task name.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
A (report, lines) pair: the JSON-ready report and its tab-separated text, biggest
|
||||||
|
first. Name tables end in a TOTAL row over all their tasks; kind tables bucket each
|
||||||
|
occurrence into <0.25, <0.5, <1, <2 or >=2 s.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
import ansible_log
|
||||||
|
|
||||||
|
BUCKETS = ((0.25, "<0.25"), (0.5, "<0.5"), (1.0, "<1"), (2.0, "<2"))
|
||||||
|
LAST_BUCKET = ">=2"
|
||||||
|
LABELS = (*(label for _, label in BUCKETS), LAST_BUCKET)
|
||||||
|
|
||||||
|
Found = list[ansible_log.Task]
|
||||||
|
|
||||||
|
|
||||||
|
def bucket(seconds: float) -> str:
|
||||||
|
return next((label for limit, label in BUCKETS if seconds < limit), LAST_BUCKET)
|
||||||
|
|
||||||
|
|
||||||
|
def ranked(rows: Iterable[dict]) -> list[dict]:
|
||||||
|
return sorted(rows, key=lambda row: -row["sum"])
|
||||||
|
|
||||||
|
|
||||||
|
def of_kind(found: Found, kind: str) -> Found:
|
||||||
|
return [item for item in found if item.kind == kind]
|
||||||
|
|
||||||
|
|
||||||
|
def names(source: str, found: Found, top: int | None) -> tuple[dict, list[str]]:
|
||||||
|
table: dict[str, dict] = {}
|
||||||
|
for item in found:
|
||||||
|
row = table.setdefault(item.name, {"name": item.name, "sum": 0.0, "count": 0})
|
||||||
|
row["sum"] += item.seconds
|
||||||
|
row["count"] += 1
|
||||||
|
rows = ranked(table.values())[:top]
|
||||||
|
total = sum(item.seconds for item in found)
|
||||||
|
report = {
|
||||||
|
"source": source,
|
||||||
|
"rows": [{**row, "sum": round(row["sum"], 3)} for row in rows],
|
||||||
|
"total": {"sum": round(total, 3), "count": len(found)},
|
||||||
|
}
|
||||||
|
lines = [f"{row['sum']:.1f}\t{row['count']}\t{row['name']}" for row in rows]
|
||||||
|
return report, [*lines, f"{total:.1f}\t{len(found)}\tTOTAL"]
|
||||||
|
|
||||||
|
|
||||||
|
def kind_table(source: str, found: Found, top: int | None) -> tuple[dict, list[str]]:
|
||||||
|
table: dict[str, dict] = {}
|
||||||
|
for item in found:
|
||||||
|
row = table.setdefault(
|
||||||
|
item.kind, {"kind": item.kind, "count": 0, "sum": 0.0, "buckets": dict.fromkeys(LABELS, 0)}
|
||||||
|
)
|
||||||
|
row["count"] += 1
|
||||||
|
row["sum"] += item.seconds
|
||||||
|
row["buckets"][bucket(item.seconds)] += 1
|
||||||
|
rows = ranked(table.values())[:top]
|
||||||
|
report = {
|
||||||
|
"source": source,
|
||||||
|
"kinds": [
|
||||||
|
{**row, "sum": round(row["sum"], 3), "mean": round(row["sum"] / row["count"], 4)} for row in rows
|
||||||
|
],
|
||||||
|
}
|
||||||
|
lines = ["\t".join(("kind", "count", "sum", "mean", *LABELS))]
|
||||||
|
lines += [
|
||||||
|
"\t".join(
|
||||||
|
(
|
||||||
|
row["kind"],
|
||||||
|
str(row["count"]),
|
||||||
|
f"{row['sum']:.1f}",
|
||||||
|
f"{row['sum'] / row['count']:.4f}",
|
||||||
|
*(str(row["buckets"][label]) for label in LABELS),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
return report, lines
|
||||||
|
|
||||||
|
|
||||||
|
def occurrences(source: str, found: Found, needle: str, top: int | None) -> tuple[dict, list[str]]:
|
||||||
|
hits = sorted((item for item in found if needle in item.name), key=lambda item: -item.seconds)
|
||||||
|
shown = hits[:top]
|
||||||
|
report = {
|
||||||
|
"source": source,
|
||||||
|
"needle": needle,
|
||||||
|
"count": len(hits),
|
||||||
|
"sum": round(sum(item.seconds for item in hits), 3),
|
||||||
|
"occurrences": [
|
||||||
|
{"line": item.line, "seconds": round(item.seconds, 3), "kind": item.kind, "name": item.name}
|
||||||
|
for item in shown
|
||||||
|
],
|
||||||
|
}
|
||||||
|
return report, [f"{item.line}\t{item.seconds:.3f}\t{item.kind}\t{item.name}" for item in shown]
|
||||||
+147
@@ -0,0 +1,147 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Measure where an Ansible run spends its time: from its log, its controller profiles or a bench.
|
||||||
|
|
||||||
|
Subcommands:
|
||||||
|
durations: per task name sum and count of the profile_tasks `(h:mm:ss.sss)` durations.
|
||||||
|
gaps: per task name sum and count of the wall time between the headers of a log
|
||||||
|
whose lines start with an ISO timestamp.
|
||||||
|
kinds: per kind (ran, skipped, included, other) count, sum, mean and the buckets
|
||||||
|
<0.25, <0.5, <1, <2 and >=2 s.
|
||||||
|
kind: per task name sum and count of one --kind.
|
||||||
|
task: every occurrence of the task names holding --needle: start line, seconds, kind.
|
||||||
|
profile-report: the main and worker profiles of --dir merged, by cumulative and own time.
|
||||||
|
bench-carrier: a synthetic JSON fact of exactly --kb KiB written to --out.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
--log: the Ansible output to read.
|
||||||
|
--from, --to: first and last line to read, 1-based and inclusive; the whole log when omitted.
|
||||||
|
--top: rows per table; every row when omitted for a log, required for profile-report.
|
||||||
|
--json: JSON instead of tab-separated text.
|
||||||
|
--source: wall (ISO timestamps) or profile (profile_tasks) for kinds, kind and task;
|
||||||
|
wall when a header in range carries a timestamp, else profile, when omitted.
|
||||||
|
--kind: ran, skipped, included or other.
|
||||||
|
--needle: substring of the task name.
|
||||||
|
--dir: profile directory; $ANSIBLE_PROFILE_DIR, else <TMPDIR>/ansible-profile, when omitted.
|
||||||
|
--kb: size of the synthetic fact in KiB.
|
||||||
|
--out: file the synthetic fact goes to.
|
||||||
|
|
||||||
|
Exit codes:
|
||||||
|
0 success, 2 usage error or nothing measurable in the range.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from collections.abc import Callable
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Trip-wire: python3 -I keeps the script directory off sys.path, and the subcommands live there.
|
||||||
|
sys.path.append(str(Path(__file__).resolve().parent))
|
||||||
|
|
||||||
|
import ansible_bench
|
||||||
|
import ansible_log
|
||||||
|
import ansible_profile
|
||||||
|
import ansible_tables
|
||||||
|
|
||||||
|
Report = tuple[dict, list[str]]
|
||||||
|
|
||||||
|
|
||||||
|
def parsed(args: argparse.Namespace, source: str | None) -> tuple[str, ansible_tables.Found]:
|
||||||
|
return ansible_log.tasks(args.log, args.first, args.last, source)
|
||||||
|
|
||||||
|
|
||||||
|
def durations(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_tables.names(*parsed(args, "profile"), args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def gaps(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_tables.names(*parsed(args, "wall"), args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def kinds(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_tables.kind_table(*parsed(args, args.source), args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def kind(args: argparse.Namespace) -> Report:
|
||||||
|
source, found = parsed(args, args.source)
|
||||||
|
return ansible_tables.names(source, ansible_tables.of_kind(found, args.kind), args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def task(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_tables.occurrences(*parsed(args, args.source), args.needle, args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def profile_report(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_profile.report(args.dir, args.top)
|
||||||
|
|
||||||
|
|
||||||
|
def bench_carrier(args: argparse.Namespace) -> Report:
|
||||||
|
return ansible_bench.write(args.kb, args.out)
|
||||||
|
|
||||||
|
|
||||||
|
COMMANDS: dict[str, Callable[[argparse.Namespace], Report]] = {
|
||||||
|
"durations": durations,
|
||||||
|
"gaps": gaps,
|
||||||
|
"kinds": kinds,
|
||||||
|
"kind": kind,
|
||||||
|
"task": task,
|
||||||
|
"profile-report": profile_report,
|
||||||
|
"bench-carrier": bench_carrier,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def positive(text: str) -> int:
|
||||||
|
try:
|
||||||
|
number = int(text)
|
||||||
|
except ValueError:
|
||||||
|
raise argparse.ArgumentTypeError(f"{text!r}: expected a whole number") from None
|
||||||
|
if number < 1:
|
||||||
|
raise argparse.ArgumentTypeError(f"{text!r} is below 1")
|
||||||
|
return number
|
||||||
|
|
||||||
|
|
||||||
|
def parse(argv: list[str]) -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||||
|
commands = parser.add_subparsers(dest="command", required=True)
|
||||||
|
output = argparse.ArgumentParser(add_help=False)
|
||||||
|
output.add_argument("--json", action="store_true")
|
||||||
|
log = argparse.ArgumentParser(add_help=False, parents=[output])
|
||||||
|
log.add_argument("--log", required=True, type=Path)
|
||||||
|
log.add_argument("--from", dest="first", type=positive)
|
||||||
|
log.add_argument("--to", dest="last", type=positive)
|
||||||
|
log.add_argument("--top", type=positive)
|
||||||
|
detect = argparse.ArgumentParser(add_help=False, parents=[log])
|
||||||
|
detect.add_argument("--source", choices=ansible_log.SOURCES)
|
||||||
|
commands.add_parser("durations", parents=[log])
|
||||||
|
commands.add_parser("gaps", parents=[log])
|
||||||
|
commands.add_parser("kinds", parents=[detect])
|
||||||
|
commands.add_parser("kind", parents=[detect]).add_argument("--kind", required=True, choices=ansible_log.KINDS)
|
||||||
|
commands.add_parser("task", parents=[detect]).add_argument("--needle", required=True)
|
||||||
|
report = commands.add_parser("profile-report", parents=[output])
|
||||||
|
report.add_argument("--dir", type=Path)
|
||||||
|
report.add_argument("--top", required=True, type=positive)
|
||||||
|
bench = commands.add_parser("bench-carrier", parents=[output])
|
||||||
|
bench.add_argument("--kb", required=True, type=positive)
|
||||||
|
bench.add_argument("--out", required=True, type=Path)
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
first, last = getattr(args, "first", None), getattr(args, "last", None)
|
||||||
|
if first is not None and last is not None and first > last:
|
||||||
|
parser.error(f"--from {first} is after --to {last}")
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str]) -> int:
|
||||||
|
args = parse(argv)
|
||||||
|
try:
|
||||||
|
report, lines = COMMANDS[args.command](args)
|
||||||
|
except (ansible_log.Unmeasurable, OSError) as error:
|
||||||
|
print(f"ansible_tasks: {error}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
print(json.dumps(report, indent=2) if args.json else "\n".join(lines))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv[1:]))
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
"""Profile ansible-playbook and every worker it forks with cProfile, once on PYTHONPATH.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
ANSIBLE_PROFILE_DIR: directory for main-<pid>.prof and worker-<pid>.prof;
|
||||||
|
<TMPDIR>/ansible-profile when unset.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
Only a process whose argv[0] is named ansible-playbook is profiled. The controller
|
||||||
|
writes its file at exit, each forked worker when it leaves through os._exit.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import atexit
|
||||||
|
import cProfile
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
PROGRAM = "ansible-playbook"
|
||||||
|
|
||||||
|
|
||||||
|
def profile_dir() -> str:
|
||||||
|
# Trip-wire: TMPDIR is the one scratch directory every sandbox lets the run write to.
|
||||||
|
return os.environ.get("ANSIBLE_PROFILE_DIR") or os.path.join(tempfile.gettempdir(), "ansible-profile")
|
||||||
|
|
||||||
|
|
||||||
|
def start(out: str) -> None:
|
||||||
|
os.makedirs(out, exist_ok=True)
|
||||||
|
controller = cProfile.Profile()
|
||||||
|
|
||||||
|
def dump_controller() -> None:
|
||||||
|
controller.disable()
|
||||||
|
controller.dump_stats(os.path.join(out, f"main-{os.getpid()}.prof"))
|
||||||
|
|
||||||
|
def after_fork() -> None:
|
||||||
|
# Trip-wire: the fork inherits the enabled controller profiler; no second one starts beside it.
|
||||||
|
controller.disable()
|
||||||
|
worker = cProfile.Profile()
|
||||||
|
real_exit = os._exit
|
||||||
|
|
||||||
|
def exit_dumping(code: int) -> None:
|
||||||
|
worker.disable()
|
||||||
|
worker.dump_stats(os.path.join(out, f"worker-{os.getpid()}.prof"))
|
||||||
|
real_exit(code)
|
||||||
|
|
||||||
|
# Trip-wire: multiprocessing workers leave through os._exit, which skips atexit.
|
||||||
|
os._exit = exit_dumping
|
||||||
|
worker.enable()
|
||||||
|
|
||||||
|
atexit.register(dump_controller)
|
||||||
|
os.register_at_fork(after_in_child=after_fork)
|
||||||
|
controller.enable()
|
||||||
|
|
||||||
|
|
||||||
|
if os.path.basename(sys.argv[0] if sys.argv else "") == PROGRAM:
|
||||||
|
start(profile_dir())
|
||||||
@@ -41,6 +41,7 @@ operator to decide, never applied as a tuning.
|
|||||||
beats a 60 s download paid once.
|
beats a 60 s download paid once.
|
||||||
- **The baseline is the newest log of the current code**; an older one may
|
- **The baseline is the newest log of the current code**; an older one may
|
||||||
carry a step since replaced, and a saving against it is fiction.
|
carry a step since replaced, and a saving against it is fiction.
|
||||||
|
- **Ansible**: measure with the `kvb-check-ansible` skill first.
|
||||||
|
|
||||||
## Coordinate every cost cluster
|
## Coordinate every cost cluster
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,125 @@
|
|||||||
|
"""Shared paths, script modules, synthetic logs and the CLI test case of the kvb-check-ansible tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SKILL = Path(__file__).resolve().parents[1] / "skills" / "kvb-check-ansible"
|
||||||
|
sys.path.insert(0, str(SKILL / "scripts"))
|
||||||
|
|
||||||
|
import ansible_bench
|
||||||
|
import ansible_log
|
||||||
|
import ansible_tables
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"BENCH",
|
||||||
|
"MIDNIGHT_LOG",
|
||||||
|
"PROFILE_DIR",
|
||||||
|
"PROFILE_LOG",
|
||||||
|
"SCRIPT",
|
||||||
|
"WALL_KINDS",
|
||||||
|
"WALL_LOG",
|
||||||
|
"LogCase",
|
||||||
|
"ansible_bench",
|
||||||
|
"ansible_log",
|
||||||
|
"ansible_tables",
|
||||||
|
]
|
||||||
|
|
||||||
|
SCRIPT = SKILL / "scripts" / "ansible_tasks.py"
|
||||||
|
PROFILE_DIR = SKILL / "scripts" / "profile"
|
||||||
|
BENCH = SKILL / "bench"
|
||||||
|
STARS = " " + "*" * 40
|
||||||
|
|
||||||
|
WALL_LOG = "\n".join(
|
||||||
|
[
|
||||||
|
"2026-11-07T06:45:40.0000000Z PLAY [deploy]" + STARS,
|
||||||
|
"2026-11-07T06:45:40.5000000Z TASK [setup : install]" + STARS,
|
||||||
|
"2026-11-07T06:45:43.0000000Z \x1b[0;33mchanged: [host]\x1b[0m",
|
||||||
|
"2026-11-07T06:45:44.0000000Z TASK [setup : check]" + STARS,
|
||||||
|
"2026-11-07T06:45:44.1000000Z skipping: [host]",
|
||||||
|
"2026-11-07T06:45:44.2Z TASK [setup : install]" + STARS,
|
||||||
|
"2026-11-07T06:45:44.5000000Z skipping: [host] => (item=a)",
|
||||||
|
"2026-11-07T06:45:45.0000000Z ok: [host] => (item=b)",
|
||||||
|
"2026-11-07T06:45:45.700000000Z TASK [setup : include]" + STARS,
|
||||||
|
"2026-11-07T06:45:45.8000000Z included: /x/y.yml for host",
|
||||||
|
"2026-11-07T06:45:46.0 TASK [setup : flush]" + STARS,
|
||||||
|
"2026-11-07T06:45:46.5000000Z [WARNING]: nothing to flush",
|
||||||
|
"2026-11-07T06:45:46.8000000Z RUNNING HANDLER [setup : restart]" + STARS,
|
||||||
|
"2026-11-07T06:45:48.0000000Z fatal: [host]: FAILED! => {}",
|
||||||
|
"2026-11-07T06:45:48.0000000Z PLAY RECAP" + STARS,
|
||||||
|
"2026-11-07T06:45:48.0000000Z host : ok=2 changed=1 skipped=1 failed=1",
|
||||||
|
]
|
||||||
|
) + "\n"
|
||||||
|
|
||||||
|
PROFILE_LINE = "Saturday 07 November 2026 06:45:40 +0000 ({}) 0:00:00.020 " + "*" * 20
|
||||||
|
PROFILE_LOG = "\n".join(
|
||||||
|
[
|
||||||
|
"PLAY [deploy]" + STARS,
|
||||||
|
"",
|
||||||
|
"TASK [Gathering Facts]" + STARS,
|
||||||
|
PROFILE_LINE.format("0:00:00.020"),
|
||||||
|
"ok: [host]",
|
||||||
|
"TASK [setup : install]" + STARS,
|
||||||
|
PROFILE_LINE.format("0:00:02.000"),
|
||||||
|
"changed: [host]",
|
||||||
|
"TASK [setup : check]" + STARS,
|
||||||
|
PROFILE_LINE.format("0:00:03.500"),
|
||||||
|
"skipping: [host]",
|
||||||
|
"TASK [setup : install]" + STARS,
|
||||||
|
PROFILE_LINE.format("0:00:00.100"),
|
||||||
|
"included: /x/y.yml for host",
|
||||||
|
"RUNNING HANDLER [setup : restart]" + STARS,
|
||||||
|
PROFILE_LINE.format("0:01:01.500"),
|
||||||
|
"ok: [host]",
|
||||||
|
"PLAY RECAP" + STARS,
|
||||||
|
"host : ok=3 changed=1 skipped=1",
|
||||||
|
"TASKS RECAP" + STARS,
|
||||||
|
PROFILE_LINE.format("0:00:00.400"),
|
||||||
|
"=" * 40,
|
||||||
|
"setup : install ----------------------- 63.50s",
|
||||||
|
]
|
||||||
|
) + "\n"
|
||||||
|
|
||||||
|
MIDNIGHT_LOG = chr(0xFEFF) + "2026-11-07T23:59:59.5Z TASK [late]" + STARS + "\n2026-11-08T00:00:00.25Z PLAY RECAP" + STARS + "\n"
|
||||||
|
|
||||||
|
WALL_KINDS = [
|
||||||
|
"kind\tcount\tsum\tmean\t<0.25\t<0.5\t<1\t<2\t>=2",
|
||||||
|
"ran\t3\t6.2\t2.0667\t0\t0\t0\t2\t1",
|
||||||
|
"other\t1\t0.8\t0.8000\t0\t0\t1\t0\t0",
|
||||||
|
"included\t1\t0.3\t0.3000\t0\t1\t0\t0\t0",
|
||||||
|
"skipped\t1\t0.2\t0.2000\t1\t0\t0\t0\t0",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class LogCase(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(tmp.cleanup)
|
||||||
|
self.base = Path(tmp.name)
|
||||||
|
self.wall = self.write("wall.log", WALL_LOG)
|
||||||
|
self.profile = self.write("profile.log", PROFILE_LOG)
|
||||||
|
|
||||||
|
def write(self, name: str, text: str) -> Path:
|
||||||
|
path = self.base / name
|
||||||
|
path.write_text(text, encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
def run_script(self, *args: str, env: dict | None = None) -> subprocess.CompletedProcess:
|
||||||
|
return subprocess.run(
|
||||||
|
[sys.executable, str(SCRIPT), *args], capture_output=True, text=True, check=False, env=env
|
||||||
|
)
|
||||||
|
|
||||||
|
def lines(self, *args: str) -> list[str]:
|
||||||
|
result = self.run_script(*args)
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
return result.stdout.splitlines()
|
||||||
|
|
||||||
|
def report(self, *args: str) -> dict:
|
||||||
|
result = self.run_script(*args, "--json")
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
return json.loads(result.stdout)
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
"""Exercise kvb-check-ansible's bench: the synthetic carrier fact and a real run of the bench playbook."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ansible_support import BENCH, SCRIPT, LogCase, ansible_bench, ansible_log
|
||||||
|
|
||||||
|
|
||||||
|
class TestBenchCarrier(LogCase):
|
||||||
|
def test_writes_exactly_the_asked_size(self):
|
||||||
|
out = self.base / "carrier.json"
|
||||||
|
result = self.run_script("bench-carrier", "--kb", "3", "--out", str(out))
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertEqual(out.stat().st_size, 3 * 1024)
|
||||||
|
fact = json.loads(out.read_text(encoding="ascii"))
|
||||||
|
self.assertEqual(result.stdout, f"{out}\t3072 bytes\t{len(fact) - 1} entries\n")
|
||||||
|
self.assertIn("pad", fact)
|
||||||
|
for size in (1024, 1500, 400 * 1024):
|
||||||
|
with self.subTest(size=size):
|
||||||
|
self.assertEqual(len(json.dumps(ansible_bench.carrier(size))), size)
|
||||||
|
self.assertGreater(len(ansible_bench.carrier(400 * 1024)), 100)
|
||||||
|
|
||||||
|
def test_bench_inventory_pins_no_interpreter_path(self):
|
||||||
|
self.assertNotRegex((BENCH / "hosts.yml").read_text(encoding="utf-8"), r"interpreter:\s*/")
|
||||||
|
|
||||||
|
|
||||||
|
@unittest.skipUnless(shutil.which("ansible-playbook"), "ansible-playbook not installed")
|
||||||
|
class TestBench(unittest.TestCase):
|
||||||
|
@classmethod
|
||||||
|
def setUpClass(cls):
|
||||||
|
tmp = tempfile.TemporaryDirectory()
|
||||||
|
cls.addClassCleanup(tmp.cleanup)
|
||||||
|
base = Path(tmp.name)
|
||||||
|
carrier = base / "carrier.json"
|
||||||
|
carrier.write_text(json.dumps(ansible_bench.carrier(8 * 1024)), encoding="ascii")
|
||||||
|
cls.log = base / "bench.log"
|
||||||
|
env = {
|
||||||
|
"PATH": os.environ["PATH"],
|
||||||
|
"HOME": str(base),
|
||||||
|
"ANSIBLE_LOCAL_TEMP": str(base / "tmp"),
|
||||||
|
"ANSIBLE_NOCOLOR": "1",
|
||||||
|
"ANSIBLE_CALLBACKS_ENABLED": "ansible.posix.profile_tasks",
|
||||||
|
}
|
||||||
|
cls.result = subprocess.run(
|
||||||
|
[
|
||||||
|
"ansible-playbook", "-i", str(BENCH / "hosts.yml"), str(BENCH / "bench.yml"),
|
||||||
|
"-e", "bench_count=3", "-e", f"bench_carrier_file={carrier}",
|
||||||
|
],
|
||||||
|
cwd=base, env=env, capture_output=True, text=True, check=False,
|
||||||
|
)
|
||||||
|
cls.log.write_text(cls.result.stdout, encoding="utf-8")
|
||||||
|
|
||||||
|
def test_runs_one_skipped_and_one_ran_task_per_item(self):
|
||||||
|
self.assertEqual(self.result.returncode, 0, self.result.stdout + self.result.stderr)
|
||||||
|
self.assertEqual(self.result.stdout.count("skipping: [bench]"), 3)
|
||||||
|
self.assertEqual(self.result.stdout.count("included: "), 3)
|
||||||
|
|
||||||
|
def test_profile_tasks_output_ranks_by_kind(self):
|
||||||
|
if ansible_log.ELAPSED.search(self.result.stdout) is None:
|
||||||
|
self.skipTest("ansible.posix.profile_tasks not installed")
|
||||||
|
result = subprocess.run(
|
||||||
|
[sys.executable, str(SCRIPT), "kinds", "--log", str(self.log), "--json"],
|
||||||
|
capture_output=True, text=True, check=False,
|
||||||
|
)
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
counts = {row["kind"]: row["count"] for row in json.loads(result.stdout)["kinds"]}
|
||||||
|
self.assertEqual(counts, {"ran": 4, "skipped": 3, "included": 1})
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
"""Exercise kvb-check-ansible's log parsing: timestamps, sources, line ranges and the task boundaries."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import calendar
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from ansible_support import MIDNIGHT_LOG, PROFILE_LOG, LogCase, ansible_log
|
||||||
|
|
||||||
|
|
||||||
|
class TestStamps(unittest.TestCase):
|
||||||
|
def test_fraction_digits_zone_and_missing_z(self):
|
||||||
|
base = calendar.timegm((2026, 11, 7, 6, 45, 47))
|
||||||
|
for text, fraction in (
|
||||||
|
("2026-11-07T06:45:47.1234567Z x", 0.1234567),
|
||||||
|
("2026-11-07T06:45:47.1Z x", 0.1),
|
||||||
|
("2026-11-07T06:45:47Z x", 0.0),
|
||||||
|
("2026-11-07T06:45:47.123456789 x", 0.123456789),
|
||||||
|
("2026-11-07T07:45:47.5+01:00 x", 0.5),
|
||||||
|
("2026-11-07T01:45:47.5-0500 x", 0.5),
|
||||||
|
("2026-11-07 06:45:47,25 x", 0.25),
|
||||||
|
):
|
||||||
|
with self.subTest(text=text):
|
||||||
|
stamp, content = ansible_log.split_stamp(text)
|
||||||
|
self.assertAlmostEqual(stamp, base + fraction, places=5)
|
||||||
|
self.assertEqual(content, "x")
|
||||||
|
self.assertEqual(ansible_log.split_stamp("TASK [x] ***"), (None, "TASK [x] ***"))
|
||||||
|
|
||||||
|
|
||||||
|
class TestWallParsing(LogCase):
|
||||||
|
def test_line_range_drops_the_task_its_end_cuts_off(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("gaps", "--log", str(self.wall), "--from", "4", "--to", "13"),
|
||||||
|
[
|
||||||
|
"1.5\t1\tsetup : install",
|
||||||
|
"0.8\t1\tsetup : flush",
|
||||||
|
"0.3\t1\tsetup : include",
|
||||||
|
"0.2\t1\tsetup : check",
|
||||||
|
"2.8\t4\tTOTAL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_midnight_bom_and_recap_close_the_last_task(self):
|
||||||
|
midnight = self.write("midnight.log", MIDNIGHT_LOG)
|
||||||
|
self.assertEqual(self.report("gaps", "--log", str(midnight))["rows"], [{"name": "late", "sum": 0.75, "count": 1}])
|
||||||
|
|
||||||
|
|
||||||
|
class TestProfileParsing(LogCase):
|
||||||
|
def test_durations_pair_each_header_with_the_next_elapsed_line(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("durations", "--log", str(self.profile)),
|
||||||
|
[
|
||||||
|
"65.0\t2\tsetup : install",
|
||||||
|
"2.0\t1\tGathering Facts",
|
||||||
|
"0.4\t1\tsetup : restart",
|
||||||
|
"0.1\t1\tsetup : check",
|
||||||
|
"67.5\t5\tTOTAL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_kinds_fall_back_to_the_profile_source(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("kinds", "--log", str(self.profile)),
|
||||||
|
[
|
||||||
|
"kind\tcount\tsum\tmean\t<0.25\t<0.5\t<1\t<2\t>=2",
|
||||||
|
"included\t1\t61.5\t61.5000\t0\t0\t0\t0\t1",
|
||||||
|
"ran\t3\t5.9\t1.9667\t0\t1\t0\t0\t2",
|
||||||
|
"skipped\t1\t0.1\t0.1000\t1\t0\t0\t0\t0",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.assertEqual(self.report("kinds", "--log", str(self.profile))["source"], "profile")
|
||||||
|
|
||||||
|
def test_profile_source_reads_a_timestamped_log_too(self):
|
||||||
|
stamped = self.write("stamped.log", "".join(f"2026-11-07T06:45:40.1234Z {line}\n" for line in PROFILE_LOG.splitlines()))
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("kind", "--log", str(stamped), "--source", "profile", "--kind", "skipped"),
|
||||||
|
["0.1\t1\tsetup : check", "0.1\t1\tTOTAL"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestUnmeasurable(LogCase):
|
||||||
|
def assertFails(self, needle: str, *args: str):
|
||||||
|
result = self.run_script(*args)
|
||||||
|
self.assertEqual((result.returncode, result.stdout), (2, ""))
|
||||||
|
self.assertIn(needle, result.stderr)
|
||||||
|
|
||||||
|
def test_wrong_source_or_range_exits_2(self):
|
||||||
|
self.assertFails("ISO timestamp", "gaps", "--log", str(self.profile))
|
||||||
|
self.assertFails("ISO timestamp", "kinds", "--log", str(self.profile), "--source", "wall")
|
||||||
|
self.assertFails("profile_tasks", "durations", "--log", str(self.wall))
|
||||||
|
self.assertFails("no task closed", "gaps", "--log", str(self.wall), "--from", "2", "--to", "2")
|
||||||
|
self.assertFails("No such file", "gaps", "--log", str(self.base / "missing.log"))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
"""Exercise kvb-check-ansible's controller profiler and its profile-report subcommand."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import cProfile
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ansible_support import PROFILE_DIR, LogCase
|
||||||
|
|
||||||
|
FORKING = (
|
||||||
|
"import multiprocessing\n"
|
||||||
|
"def work():\n"
|
||||||
|
" return sum(range(1000))\n"
|
||||||
|
"if __name__ == '__main__':\n"
|
||||||
|
" child = multiprocessing.get_context('fork').Process(target=work)\n"
|
||||||
|
" child.start()\n"
|
||||||
|
" child.join()\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def busy() -> int:
|
||||||
|
return sum(number * number for number in range(20000))
|
||||||
|
|
||||||
|
|
||||||
|
class TestProfileReport(LogCase):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.profiles = self.base / "profiles"
|
||||||
|
self.profiles.mkdir()
|
||||||
|
for name in ("main-1.prof", "worker-2.prof", "worker-3.prof"):
|
||||||
|
profiler = cProfile.Profile()
|
||||||
|
profiler.enable()
|
||||||
|
busy()
|
||||||
|
profiler.disable()
|
||||||
|
profiler.dump_stats(self.profiles / name)
|
||||||
|
|
||||||
|
def test_merges_main_and_worker_profiles(self):
|
||||||
|
found = self.report("profile-report", "--dir", str(self.profiles), "--top", "3")
|
||||||
|
self.assertEqual((found["main"]["profiles"], found["worker"]["profiles"]), (1, 2))
|
||||||
|
for role, calls in (("main", 1), ("worker", 2)):
|
||||||
|
with self.subTest(role=role):
|
||||||
|
self.assertLessEqual(len(found[role]["cumulative"]), 3)
|
||||||
|
self.assertLessEqual(len(found[role]["tottime"]), 3)
|
||||||
|
rows = [row for row in found[role]["cumulative"] if row["function"].endswith("(busy)")]
|
||||||
|
self.assertEqual([row["calls"] for row in rows], [calls])
|
||||||
|
lines = self.lines("profile-report", "--dir", str(self.profiles), "--top", "2")
|
||||||
|
self.assertRegex(lines[0], r"^== main: 1 profiles, \d+\.\d{3} s ==$")
|
||||||
|
self.assertIn("-- tottime: cumtime, tottime, calls, function --", lines)
|
||||||
|
self.assertTrue(any(line.startswith("== worker: 2 profiles") for line in lines))
|
||||||
|
|
||||||
|
def test_dir_comes_from_the_environment_and_empty_exits_2(self):
|
||||||
|
env = {"PATH": os.environ["PATH"], "ANSIBLE_PROFILE_DIR": str(self.profiles)}
|
||||||
|
result = self.run_script("profile-report", "--top", "1", env=env)
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertTrue(result.stdout.startswith("== main: 1 profiles"))
|
||||||
|
empty = self.base / "empty"
|
||||||
|
empty.mkdir()
|
||||||
|
result = self.run_script("profile-report", "--dir", str(empty), "--top", "1")
|
||||||
|
self.assertEqual(result.returncode, 2)
|
||||||
|
self.assertIn("no main-*.prof or worker-*.prof", result.stderr)
|
||||||
|
|
||||||
|
|
||||||
|
class TestProfiler(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(tmp.cleanup)
|
||||||
|
self.base = Path(tmp.name)
|
||||||
|
|
||||||
|
def fork(self, program: str, **env: str) -> None:
|
||||||
|
script = self.base / program
|
||||||
|
script.write_text(FORKING, encoding="utf-8")
|
||||||
|
result = subprocess.run(
|
||||||
|
[sys.executable, str(script)],
|
||||||
|
env={"PATH": os.environ["PATH"], "PYTHONPATH": str(PROFILE_DIR), "TMPDIR": str(self.base), **env},
|
||||||
|
capture_output=True, text=True, check=False,
|
||||||
|
)
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
|
||||||
|
def test_ansible_playbook_and_its_workers_land_under_tmpdir(self):
|
||||||
|
self.fork("ansible-playbook")
|
||||||
|
out = self.base / "ansible-profile"
|
||||||
|
self.assertEqual(len(list(out.glob("main-*.prof"))), 1)
|
||||||
|
self.assertEqual(len(list(out.glob("worker-*.prof"))), 1)
|
||||||
|
|
||||||
|
def test_profile_dir_wins_and_other_programs_stay_unprofiled(self):
|
||||||
|
self.fork("ansible-playbook", ANSIBLE_PROFILE_DIR=str(self.base / "chosen"))
|
||||||
|
self.assertEqual(len(list((self.base / "chosen").glob("*.prof"))), 2)
|
||||||
|
self.fork("ansible", ANSIBLE_PROFILE_DIR=str(self.base / "other"))
|
||||||
|
self.assertFalse((self.base / "other").exists())
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
"""Exercise kvb-check-ansible's reporting subcommands: name, kind and occurrence tables, JSON and usage errors."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from ansible_support import WALL_KINDS, LogCase, ansible_tables
|
||||||
|
|
||||||
|
|
||||||
|
class TestNameTables(LogCase):
|
||||||
|
def test_gaps_sum_per_name_biggest_first_with_total(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("gaps", "--log", str(self.wall)),
|
||||||
|
[
|
||||||
|
"5.0\t2\tsetup : install",
|
||||||
|
"1.2\t1\tsetup : restart",
|
||||||
|
"0.8\t1\tsetup : flush",
|
||||||
|
"0.3\t1\tsetup : include",
|
||||||
|
"0.2\t1\tsetup : check",
|
||||||
|
"7.5\t6\tTOTAL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_gaps_json_and_top_keep_the_full_total(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.report("gaps", "--log", str(self.wall), "--top", "2"),
|
||||||
|
{
|
||||||
|
"source": "wall",
|
||||||
|
"rows": [
|
||||||
|
{"name": "setup : install", "sum": 5.0, "count": 2},
|
||||||
|
{"name": "setup : restart", "sum": 1.2, "count": 1},
|
||||||
|
],
|
||||||
|
"total": {"sum": 7.5, "count": 6},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_kind_sums_one_kind_per_name(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("kind", "--log", str(self.wall), "--kind", "ran"),
|
||||||
|
["5.0\t2\tsetup : install", "1.2\t1\tsetup : restart", "6.2\t3\tTOTAL"],
|
||||||
|
)
|
||||||
|
self.assertEqual(self.lines("kind", "--log", str(self.wall), "--kind", "other"), ["0.8\t1\tsetup : flush", "0.8\t1\tTOTAL"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestKindTable(LogCase):
|
||||||
|
def test_kinds_detect_ran_before_skipped_and_fill_the_buckets(self):
|
||||||
|
self.assertEqual(self.lines("kinds", "--log", str(self.wall)), WALL_KINDS)
|
||||||
|
found = self.report("kinds", "--log", str(self.wall))
|
||||||
|
self.assertEqual(found["source"], "wall")
|
||||||
|
self.assertEqual(
|
||||||
|
found["kinds"][0],
|
||||||
|
{"kind": "ran", "count": 3, "sum": 6.2, "mean": 2.0667,
|
||||||
|
"buckets": {"<0.25": 0, "<0.5": 0, "<1": 0, "<2": 2, ">=2": 1}},
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_bucket_edges(self):
|
||||||
|
for seconds, label in ((0.0, "<0.25"), (0.2499, "<0.25"), (0.25, "<0.5"), (0.5, "<1"), (1.0, "<2"), (1.999, "<2"), (2.0, ">=2")):
|
||||||
|
with self.subTest(seconds=seconds):
|
||||||
|
self.assertEqual(ansible_tables.bucket(seconds), label)
|
||||||
|
|
||||||
|
|
||||||
|
class TestOccurrences(LogCase):
|
||||||
|
def test_task_lists_every_occurrence_of_the_needle(self):
|
||||||
|
self.assertEqual(
|
||||||
|
self.lines("task", "--log", str(self.wall), "--needle", "install"),
|
||||||
|
["2\t3.500\tran\tsetup : install", "6\t1.500\tran\tsetup : install"],
|
||||||
|
)
|
||||||
|
found = self.report("task", "--log", str(self.wall), "--needle", "install", "--top", "1")
|
||||||
|
self.assertEqual(
|
||||||
|
found,
|
||||||
|
{
|
||||||
|
"source": "wall",
|
||||||
|
"needle": "install",
|
||||||
|
"count": 2,
|
||||||
|
"sum": 5.0,
|
||||||
|
"occurrences": [{"line": 2, "seconds": 3.5, "kind": "ran", "name": "setup : install"}],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestUsage(LogCase):
|
||||||
|
def test_usage_errors_exit_2(self):
|
||||||
|
for args in (
|
||||||
|
("gaps",),
|
||||||
|
("gaps", "--log", str(self.wall), "--from", "5", "--to", "4"),
|
||||||
|
("gaps", "--log", str(self.wall), "--top", "0"),
|
||||||
|
("kind", "--log", str(self.wall), "--kind", "slow"),
|
||||||
|
("task", "--log", str(self.wall)),
|
||||||
|
("profile-report", "--dir", str(self.base)),
|
||||||
|
("bench-carrier", "--kb", "0", "--out", str(self.base / "c.json")),
|
||||||
|
):
|
||||||
|
with self.subTest(args=args):
|
||||||
|
self.assertEqual(self.run_script(*args).returncode, 2)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Reference in new issue
Block a user