Merge branch 'fix/hub-cleanup-pkgmgr-local' into unify/skills-20261114
This commit is contained in:
commit
568747651b
19 files changed
+1534
-189
No files matched your search
@@ -125,6 +125,33 @@ from three views down to any agent, branch or artefact.
|
|||||||
worktree holds is skipped and left to `kvb-git-gc`, and no worktree,
|
worktree holds is skipped and left to `kvb-git-gc`, and no worktree,
|
||||||
Compose stack or volume is touched. Rejected: per Scope, also removing
|
Compose stack or volume is touched. Rejected: per Scope, also removing
|
||||||
the worktree and running `compose down` without `-v`.
|
the worktree and running `compose down` without `-v`.
|
||||||
|
- **Amended by the operator 2026-11-14**: only a branch whose local tip
|
||||||
|
the `pkgmgr branch cleanup --local` dry run deletes is offered: the tree
|
||||||
|
timer writes that dry run (`--json`, per known repository within a
|
||||||
|
budget, again only when its refs, HEAD, config, reftable or worktree
|
||||||
|
heads changed, the file rewritten after each repository) as
|
||||||
|
`local-plan.json` beside the tree file; a repository whose dry run
|
||||||
|
failed, timed out or was killed keeps its old plan marked stale, and
|
||||||
|
the hub offers nothing there and says why. The hub and the host decide
|
||||||
|
with one predicate on its item for that tip (`merged-clean`, no skip
|
||||||
|
reason, exactly the action `local-branch`); the hub also keeps the
|
||||||
|
local upstream of another branch, as pkgmgr keeps it for the host's
|
||||||
|
dry run of that branch alone. A branch that dry run keeps, such as one
|
||||||
|
merged by pull request (`pr#N`), squash or patch whose local tip is no
|
||||||
|
ancestor of the default branch, shows its dry run without a Delete
|
||||||
|
button, with the skip reason of the dry run and branch gc
|
||||||
|
(`kvb-git-gc`, `pkgmgr gc`) as the way to remove it, and the host
|
||||||
|
refuses its signed delete. The host runs one dry run and one apply per
|
||||||
|
repository and job; an item pkgmgr deleted at another merged tip than
|
||||||
|
the signed one is done with a warning, one the apply keeps is failed
|
||||||
|
with pkgmgr's skip reason, and only a branch pkgmgr reports deleted is
|
||||||
|
done: a signed branch whose ref is gone otherwise is `gone` ("deleted
|
||||||
|
outside this run" when pkgmgr answered without deleting it, "gone
|
||||||
|
after an interrupted apply" when the apply ended without output),
|
||||||
|
rewrites the tree and shows apart from deleted and failed. A dry run
|
||||||
|
or apply that names gone branches unknown runs once more without
|
||||||
|
them. Rejected: deciding by
|
||||||
|
`merged_by` alone, and offering a Delete the host then refuses.
|
||||||
22. **Organisation update**: a new skill `kvb-org-update` tells every agent
|
22. **Organisation update**: a new skill `kvb-org-update` tells every agent
|
||||||
and client which skills changed and how to use them, as one
|
and client which skills changed and how to use them, as one
|
||||||
`newsletter` per update; phase 1 announces the radio change through it.
|
`newsletter` per update; phase 1 announces the radio change through it.
|
||||||
@@ -641,23 +668,34 @@ flowchart TD
|
|||||||
- [x] The Artifacts Tree marks every branch and worktree with icons for
|
- [x] The Artifacts Tree marks every branch and worktree with icons for
|
||||||
merged (with what merged it), clean, uncommitted changes (with counts),
|
merged (with what merged it), clean, uncommitted changes (with counts),
|
||||||
unpushed commits, gone upstream, locked, stale and protected.
|
unpushed commits, gone upstream, locked, stale and protected.
|
||||||
- [x] A `merged-clean` item without a gc skip reason shows a delete button,
|
- [x] A `merged-clean` item without a gc skip reason whose local tip the
|
||||||
and the Tree header shows "Delete all merged"; both are disabled,
|
`pkgmgr branch cleanup --local` dry run in `local-plan.json` deletes
|
||||||
with the reason as tooltip, while `delete_ready` is false.
|
shows a delete button, also when its remote tip was merged by squash or
|
||||||
|
patch, and the Tree header shows "Delete all merged"; both are
|
||||||
|
disabled, with the reason as tooltip, while `delete_ready` is false.
|
||||||
|
- [x] A `merged-clean` item that dry run keeps, or that it has not planned
|
||||||
|
at this tip yet, shows no delete button in its Tree row, its popup or
|
||||||
|
the delete sheet; its dry run says `pkgmgr branch cleanup --local` keeps
|
||||||
|
it with the skip reason of that dry run and names branch gc
|
||||||
|
(`kvb-git-gc`, `pkgmgr gc`) as the way to remove it, and the unit
|
||||||
|
refuses its signed delete by the same predicate.
|
||||||
- [x] A click shows the gc dry-run lines of exactly the chosen items;
|
- [x] A click shows the gc dry-run lines of exactly the chosen items;
|
||||||
"Delete" requires the operator signature through `/api/sign`.
|
"Delete" requires the operator signature through `/api/sign`.
|
||||||
- [x] The orchestrator writes the signed job (repository, branch, worktree
|
- [x] The orchestrator writes the signed job (repository, branch, worktree
|
||||||
and tip sha per item, no command) into a host-mounted job directory and
|
and tip sha per item, no command) into a host-mounted job directory and
|
||||||
holds no git, token or key.
|
holds no git, token or key.
|
||||||
- [ ] A systemd user path unit verifies the signature on the host and runs
|
- [x] A systemd user path unit verifies the signature on the host and runs
|
||||||
`pkgmgr branch cleanup --apply --local` (the `--local` release is
|
`pkgmgr branch cleanup --apply --local` (pkgmgr 2.5.0) for exactly the
|
||||||
pending in pkgmgr) for exactly the signed items, matched exactly, never
|
signed items, matched exactly, never as a pattern; it removes the local
|
||||||
as a pattern; it removes the local branch, never a remote branch, a
|
branch, never a remote branch, a worktree, a Compose stack or a volume,
|
||||||
worktree, a Compose stack or a volume, and needs no SSH key; a branch a
|
and needs no SSH key; a branch a worktree holds is skipped and left to
|
||||||
worktree holds is skipped and left to `kvb-git-gc`.
|
`kvb-git-gc`.
|
||||||
- [x] The unit refuses, and changes nothing for, an item whose tip or state
|
- [x] The unit refuses, and changes nothing for, an item whose tip or state
|
||||||
differs from the signed one, an unknown repository, a non-`merged-clean`
|
differs from the signed one, an unknown repository, a non-`merged-clean`
|
||||||
item and a job with a bad or missing signature, and reports why.
|
item and a job with a bad or missing signature, and reports why; an
|
||||||
|
item pkgmgr deleted at another merged tip than the signed one is done
|
||||||
|
with a warning, and one it did not delete there is failed, or `gone`,
|
||||||
|
never done, when its ref is gone.
|
||||||
- [x] The unit writes each item's result back and rewrites the tree file; the
|
- [x] The unit writes each item's result back and rewrites the tree file; the
|
||||||
Tree shows the result.
|
Tree shows the result.
|
||||||
|
|
||||||
|
|||||||
@@ -52,9 +52,13 @@ changed and what it costs, and a gate means something again.
|
|||||||
labelled stale and counted apart. Rejected: count everything, only drop
|
labelled stale and counted apart. Rejected: count everything, only drop
|
||||||
operator rows.
|
operator rows.
|
||||||
10. **Clean all merged**: dry run, one passkey, signed job. Local branches
|
10. **Clean all merged**: dry run, one passkey, signed job. Local branches
|
||||||
merged by ancestry and without a remote branch need no token; remote
|
merged by ancestry and without a remote branch need no token; the local
|
||||||
branches and squash-merge candidates need the GitHub token, stored through
|
branch of one with a remote branch needs the GitHub token, stored through
|
||||||
an operator row. Rejected: wait for the token, ancestry for remotes.
|
an operator row. A branch the `pkgmgr branch cleanup --local` dry run
|
||||||
|
keeps, such as a squash or patch merge whose local tip is no ancestor of
|
||||||
|
the default branch, is never deleted here, token or not: its dry run says
|
||||||
|
why and names branch gc (`kvb-git-gc`, `pkgmgr gc`) (operator
|
||||||
|
2026-11-14). Rejected: wait for the token, ancestry for remotes.
|
||||||
11. **Ancestry**: Artifacts and Agents get a family tree (generations top down)
|
11. **Ancestry**: Artifacts and Agents get a family tree (generations top down)
|
||||||
in the slot of today's Tree view. A branch may have several parents;
|
in the slot of today's Tree view. A branch may have several parents;
|
||||||
parents are recorded at creation (post-checkout hook), older branches are
|
parents are recorded at creation (post-checkout hook), older branches are
|
||||||
@@ -227,8 +231,10 @@ flowchart TD
|
|||||||
- [x] "Clean all merged" shows the gc dry run of every `merged-clean` item; one
|
- [x] "Clean all merged" shows the gc dry run of every `merged-clean` item; one
|
||||||
passkey runs the signed job.
|
passkey runs the signed job.
|
||||||
- [ ] Without a GitHub token, local branches merged by ancestry into the
|
- [ ] Without a GitHub token, local branches merged by ancestry into the
|
||||||
default branch and without a remote branch can be deleted; the others
|
default branch and without a remote branch can be deleted; the other
|
||||||
show "token needed", and an operator row stores the token.
|
branches the `--local` dry run deletes show "token needed", and an
|
||||||
|
operator row stores the token. A branch that dry run keeps shows no
|
||||||
|
delete and no "token needed", with or without the token.
|
||||||
- [ ] Artifacts and Agents show Ancestry in the Tree slot: generations top
|
- [ ] Artifacts and Agents show Ancestry in the Tree slot: generations top
|
||||||
down, several parents per node; a derived parent edge is dashed.
|
down, several parents per node; a derived parent edge is dashed.
|
||||||
- [ ] Dragging a branch onto another stages a parent edge, applied with the
|
- [ ] Dragging a branch onto another stages a parent edge, applied with the
|
||||||
|
|||||||
+217
-66
@@ -15,18 +15,28 @@
|
|||||||
Returns:
|
Returns:
|
||||||
Runs once over every job in --jobs, as the operator on the host: per item
|
Runs once over every job in --jobs, as the operator on the host: per item
|
||||||
it verifies the operator's passkey signature over repository, branch,
|
it verifies the operator's passkey signature over repository, branch,
|
||||||
worktree and tip, then in the repository runs `pkgmgr branch cleanup
|
worktree and tip; then per repository it runs one `pkgmgr branch cleanup
|
||||||
--json --branch <name>` as a dry run and, only when that plan names
|
--local --json --branch <name>...` dry run over its signed branches and
|
||||||
exactly this branch as merged-clean without skip reason at the signed
|
one `pkgmgr branch cleanup --apply --local --json --branch <name>...`
|
||||||
tip and worktree and deletes no remote branch, `pkgmgr branch cleanup
|
over those the plan names exactly at the signed tip and worktree and
|
||||||
--apply --json --branch <name>`. While the tree file says pkgmgr has no GitHub token
|
deletes exactly the local branch of (signing.local_keeps, the predicate
|
||||||
(delete_ready false), only a branch the dry run shows merged by ancestry
|
the hub offers by); pkgmgr deletes only a local branch whose planned tip
|
||||||
into the default branch and without a remote branch is deleted; the
|
is an ancestor of the default branch, by update-ref with that tip. While
|
||||||
others are refused with "token needed". Every other item is refused and
|
the tree file says pkgmgr has no GitHub token (delete_ready false), only
|
||||||
nothing changes for it. An applied item is done only when pkgmgr's
|
a branch the dry run shows merged by ancestry into the default branch
|
||||||
result for it is done; a skipped one or one without result is failed
|
and without a remote branch is deleted; the others are refused with
|
||||||
with pkgmgr's reason. Writes each item's result, rewrites the tree file
|
"token needed". Every other item is refused and nothing changes for it.
|
||||||
after a deletion and removes the job. The exit code is 0.
|
An applied item is done only when pkgmgr's result for it is done, with a
|
||||||
|
warning when pkgmgr deleted it at another merged tip than the signed
|
||||||
|
one; one the apply keeps is failed with pkgmgr's skip reason. A signed
|
||||||
|
branch pkgmgr did not report deleted whose ref is gone is gone, never
|
||||||
|
done: "deleted outside this run" when pkgmgr answered without deleting
|
||||||
|
it (a failed result, no result, or "Unknown --branch"), "gone after an
|
||||||
|
interrupted apply" when the apply ended without JSON; a present one is
|
||||||
|
failed. When a dry run or apply answers "Unknown --branch" for gone
|
||||||
|
branches, it runs once more without them.
|
||||||
|
Writes each item's result, rewrites the tree file after a deletion or a
|
||||||
|
gone branch and removes the job. The exit code is 0.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -49,7 +59,10 @@ UNSAFE = re.compile(r"[*?\[\]\\\s\x00-\x1f\x7f]|^-|\.\.|@\{")
|
|||||||
OUTPUT_LIMIT = 16 << 10
|
OUTPUT_LIMIT = 16 << 10
|
||||||
TIMEOUT = 600
|
TIMEOUT = 600
|
||||||
DONE = "done"
|
DONE = "done"
|
||||||
REMOTE_ACTION = "remote-branch:"
|
GONE = "gone"
|
||||||
|
DELETED_OUTSIDE = "deleted outside this run"
|
||||||
|
GONE_INTERRUPTED = "gone after an interrupted apply"
|
||||||
|
UNKNOWN = re.compile(r"^Unknown --branch: (.*)$", re.MULTILINE)
|
||||||
TOKEN_NEEDED = "token needed: without the GitHub token only a local branch merged by ancestry into the default branch and without a remote branch is deleted"
|
TOKEN_NEEDED = "token needed: without the GitHub token only a local branch merged by ancestry into the default branch and without a remote branch is deleted"
|
||||||
# the cwd is a repository agents can write: its config may name hooks, and
|
# the cwd is a repository agents can write: its config may name hooks, and
|
||||||
# pkgmgr releases before 4c8b137 run git branch -D without scan mode
|
# pkgmgr releases before 4c8b137 run git branch -D without scan mode
|
||||||
@@ -84,23 +97,57 @@ def tree_facts(tree: Path) -> tuple[dict[str, str], bool]:
|
|||||||
return {}, False
|
return {}, False
|
||||||
|
|
||||||
|
|
||||||
def run(pkgmgr: Path, argv: list[str], cwd: str) -> tuple[int, str, str]:
|
def run(pkgmgr: Path, argv: list[str], cwd: str, timeout: int) -> tuple[int, str, str]:
|
||||||
|
"""Args:
|
||||||
|
pkgmgr: the pkgmgr executable.
|
||||||
|
argv: its arguments.
|
||||||
|
cwd: the repository it runs in, without hooks or fsmonitor.
|
||||||
|
timeout: seconds before pkgmgr is killed.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Exit code, stdout and the tail of stderr; 127 with the error when
|
||||||
|
pkgmgr could not start or was killed at the timeout.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
done = subprocess.run([str(pkgmgr), *argv], cwd=cwd, env={**os.environ, **git_overrides()}, stdin=subprocess.DEVNULL, capture_output=True, text=True, errors="replace", timeout=TIMEOUT, check=False)
|
done = subprocess.run([str(pkgmgr), *argv], cwd=cwd, env={**os.environ, **git_overrides()}, stdin=subprocess.DEVNULL, capture_output=True, text=True, errors="replace", timeout=timeout, check=False)
|
||||||
except (OSError, subprocess.TimeoutExpired) as error:
|
except (OSError, subprocess.TimeoutExpired) as error:
|
||||||
return 127, "", str(error)
|
return 127, "", str(error)
|
||||||
return done.returncode, done.stdout, done.stderr[-OUTPUT_LIMIT:]
|
return done.returncode, done.stdout, done.stderr[-OUTPUT_LIMIT:]
|
||||||
|
|
||||||
|
|
||||||
def plan_argv(branch: str, apply: bool) -> list[str]:
|
def branch_tip(repository: str, branch: str) -> str | None:
|
||||||
|
"""Returns:
|
||||||
|
The local branch's tip as git reads it in the repository and no
|
||||||
|
parent of it, "" when the repository has no such branch, None when
|
||||||
|
git cannot tell.
|
||||||
|
"""
|
||||||
|
env = {**os.environ, **git_overrides(), "GIT_CEILING_DIRECTORIES": str(Path(repository).parent)}
|
||||||
|
try:
|
||||||
|
done = subprocess.run(["git", "rev-parse", "--verify", "--quiet", f"refs/heads/{branch}"], cwd=repository, env=env, stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=60, check=False)
|
||||||
|
except (OSError, subprocess.TimeoutExpired):
|
||||||
|
return None
|
||||||
|
if done.returncode == 0 and TIP.fullmatch(done.stdout.strip()):
|
||||||
|
return done.stdout.strip()
|
||||||
|
return "" if done.returncode == 1 and not done.stdout.strip() else None
|
||||||
|
|
||||||
|
|
||||||
|
def unknown(stderr: str) -> set[str]:
|
||||||
|
"""Returns:
|
||||||
|
The branch names pkgmgr's "Unknown --branch: a, b" line names.
|
||||||
|
"""
|
||||||
|
return {name for found in UNKNOWN.findall(stderr) for name in found.split(", ")}
|
||||||
|
|
||||||
|
|
||||||
|
def plan_argv(branches: list[str], apply: bool) -> list[str]:
|
||||||
"""Args:
|
"""Args:
|
||||||
branch: the exact branch name.
|
branches: the exact branch names of one repository.
|
||||||
apply: execute the plan instead of the dry run.
|
apply: execute the plan instead of the dry run.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The `pkgmgr branch cleanup` arguments for this branch's plan in JSON.
|
The `pkgmgr branch cleanup --local` arguments for these branches'
|
||||||
|
plan in JSON: local branches only, no forge token, no remote delete.
|
||||||
"""
|
"""
|
||||||
return ["branch", "cleanup", *(["--apply"] if apply else []), "--json", "--branch", branch]
|
return ["branch", "cleanup", *(["--apply"] if apply else []), "--local", "--json", *[part for branch in branches for part in ("--branch", branch)]]
|
||||||
|
|
||||||
|
|
||||||
def plan_items(stdout: str) -> list[dict] | None:
|
def plan_items(stdout: str) -> list[dict] | None:
|
||||||
@@ -126,20 +173,85 @@ def shaped(item: object) -> bool:
|
|||||||
return item["repository"].startswith("/") and bool(TIP.fullmatch(item["tip"])) and (item["worktree"] == "" or item["worktree"].startswith("/"))
|
return item["repository"].startswith("/") and bool(TIP.fullmatch(item["tip"])) and (item["worktree"] == "" or item["worktree"].startswith("/"))
|
||||||
|
|
||||||
|
|
||||||
def mismatch(entry: dict, item: dict) -> str | None:
|
def tip_changed(entry: dict, item: dict) -> str | None:
|
||||||
if entry.get("state") != "merged-clean":
|
|
||||||
return f"state is {entry.get('state')}, not merged-clean"
|
|
||||||
if entry.get("skip"):
|
|
||||||
return f"gc skip reason: {entry['skip']}"
|
|
||||||
tip = entry.get("local_sha") or entry.get("head")
|
tip = entry.get("local_sha") or entry.get("head")
|
||||||
if tip != item["tip"]:
|
return None if tip == item["tip"] else f"tip changed: signed {item['tip'][:12]}, now {str(tip)[:12]}"
|
||||||
return f"tip changed: signed {item['tip'][:12]}, now {str(tip)[:12]}"
|
|
||||||
|
|
||||||
|
def mismatch(entry: dict, item: dict) -> str | None:
|
||||||
|
changed = tip_changed(entry, item)
|
||||||
|
if changed:
|
||||||
|
return changed
|
||||||
if (entry.get("worktree") or "") != item["worktree"]:
|
if (entry.get("worktree") or "") != item["worktree"]:
|
||||||
return f"worktree changed: signed {item['worktree'] or 'none'}, now {entry.get('worktree') or 'none'}"
|
return f"worktree changed: signed {item['worktree'] or 'none'}, now {entry.get('worktree') or 'none'}"
|
||||||
remote = [str(action) for action in entry.get("actions") or [] if str(action).startswith(REMOTE_ACTION)]
|
kept = signing.local_keeps(entry)
|
||||||
if remote:
|
return f"pkgmgr branch cleanup --local keeps it: {kept}" if kept else None
|
||||||
return f"the plan also deletes {', '.join(remote)[:200]}; a signed delete is local only"
|
|
||||||
return None
|
|
||||||
|
def refused(item: dict, reason: str) -> dict:
|
||||||
|
return {**item, "status": "refused", "reason": reason}
|
||||||
|
|
||||||
|
|
||||||
|
def applied_result(item: dict, applied: list[dict] | None, code: int, stderr: str) -> dict:
|
||||||
|
"""Args:
|
||||||
|
item: one signed item the apply chose.
|
||||||
|
applied: every item `--apply` printed for its repository, None when
|
||||||
|
it printed no JSON.
|
||||||
|
code: the apply's exit code, 1 when any of its items failed.
|
||||||
|
stderr: the apply's error output.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The item done only when pkgmgr's result for it is done, with a
|
||||||
|
warning as reason when pkgmgr deleted it at another tip than the
|
||||||
|
signed one, which update-ref only does for the tip pkgmgr planned as
|
||||||
|
merged; failed with pkgmgr's skip reason when the apply kept it.
|
||||||
|
Without a done result its ref decides: gone is gone, "gone after an
|
||||||
|
interrupted apply" when the apply printed no JSON and named no
|
||||||
|
unknown branch, else "deleted outside this run"; present is failed
|
||||||
|
with pkgmgr's reason.
|
||||||
|
"""
|
||||||
|
mine = next((entry for entry in applied or [] if entry.get("branch") == item["branch"]), {})
|
||||||
|
outcome = mine.get("result")
|
||||||
|
moved = tip_changed(mine, item) if mine else None
|
||||||
|
if isinstance(outcome, str) and outcome.split(";")[0] == DONE:
|
||||||
|
warning = f"deleted at another merged tip than the signed dry run: {moved}" if moved else ""
|
||||||
|
return {**item, "status": DONE, "reason": warning, "code": code, "applied": [mine]}
|
||||||
|
if mine.get("skip") and not outcome:
|
||||||
|
reason = f"pkgmgr branch cleanup --local keeps it: {mine['skip']}"
|
||||||
|
elif moved:
|
||||||
|
reason = f"the apply planned another tip than the signed dry run: {moved}"
|
||||||
|
elif not (isinstance(outcome, str) and outcome.startswith("skipped")) and branch_tip(item["repository"], item["branch"]) == "":
|
||||||
|
why = GONE_INTERRUPTED if applied is None and not unknown(stderr) else DELETED_OUTSIDE
|
||||||
|
return {**item, "status": GONE, "reason": why, "code": code, "applied": [mine] if mine else []}
|
||||||
|
else:
|
||||||
|
reason = outcome if isinstance(outcome, str) and outcome else stderr.strip()[-500:] or f"pkgmgr reported no result for {item['branch']}"
|
||||||
|
return {**item, "status": "failed", "reason": reason[:500], "code": code, "applied": [mine] if mine else []}
|
||||||
|
|
||||||
|
|
||||||
|
def screened(items: list[dict], plan: list[dict], default: str, ready: bool) -> dict[str, dict]:
|
||||||
|
"""Args:
|
||||||
|
items: admitted items of one repository, each branch once.
|
||||||
|
plan: the items of their dry run.
|
||||||
|
default: the repository's default branch.
|
||||||
|
ready: whether pkgmgr holds the GitHub token.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The refusal by branch of each item the plan does not delete exactly
|
||||||
|
at its signed tip and worktree, or without the token not token-free;
|
||||||
|
of every item when the plan names other branches too.
|
||||||
|
"""
|
||||||
|
others = sorted({str(entry.get("branch")) for entry in plan} - {item["branch"] for item in items})
|
||||||
|
if others:
|
||||||
|
return {item["branch"]: refused(item, f"the dry run names other branches too, a pattern match: {', '.join(others)[:500]}") for item in items}
|
||||||
|
refusals = {}
|
||||||
|
for item in items:
|
||||||
|
mine = [entry for entry in plan if entry.get("branch") == item["branch"]]
|
||||||
|
reason = mismatch(mine[0], item) if len(mine) == 1 else f"the dry run names {len(mine)} items for {item['branch']}"
|
||||||
|
if reason is None and not ready and not signing.token_free(mine[0], default):
|
||||||
|
reason = TOKEN_NEEDED
|
||||||
|
if reason is not None:
|
||||||
|
refusals[item["branch"]] = refused(item, reason)
|
||||||
|
return refusals
|
||||||
|
|
||||||
|
|
||||||
class Cleanup:
|
class Cleanup:
|
||||||
@@ -148,53 +260,78 @@ class Cleanup:
|
|||||||
self.tree = tree
|
self.tree = tree
|
||||||
self.pkgmgr = pkgmgr
|
self.pkgmgr = pkgmgr
|
||||||
|
|
||||||
def item(self, item: dict, proof: object, known: dict[str, str], ready: bool, now: float) -> dict:
|
def admitted(self, item: dict, proof: object, known: dict[str, str], now: float) -> str | None:
|
||||||
"""Args:
|
"""Args:
|
||||||
item: one signed item with repository, branch, worktree and tip.
|
item: one signed item with repository, branch, worktree and tip.
|
||||||
proof: the job's {"payload", "assertion"}.
|
proof: the job's {"payload", "assertion"}.
|
||||||
known: the repositories the tree file says pkgmgr knows, each
|
known: the repositories the tree file says pkgmgr knows.
|
||||||
with its default branch.
|
|
||||||
ready: whether pkgmgr holds the GitHub token; without it only a
|
|
||||||
branch signing.token_free() accepts in the dry run is deleted.
|
|
||||||
now: the current time in epoch seconds.
|
now: the current time in epoch seconds.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The item with status done, refused or failed and the reason;
|
Why the item is refused before any dry run; None when the
|
||||||
refused items changed nothing, failed ones are those pkgmgr did
|
operator's signature lists it, pkgmgr knows its repository and
|
||||||
not report done.
|
its branch is an exact name.
|
||||||
"""
|
"""
|
||||||
def refused(reason: str) -> dict:
|
|
||||||
return {**item, "status": "refused", "reason": reason}
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
verifier.check(proof, signing.cleanup_item(item["repository"], item["branch"], item["worktree"], item["tip"]), "", self.trust, now)
|
verifier.check(proof, signing.cleanup_item(item["repository"], item["branch"], item["worktree"], item["tip"]), "", self.trust, now)
|
||||||
except (signing.Refused, OSError) as error:
|
except (signing.Refused, OSError) as error:
|
||||||
return refused(f"no valid operator signature: {error}")
|
return f"no valid operator signature: {error}"
|
||||||
if item["repository"] not in known or not Path(item["repository"]).is_dir():
|
if item["repository"] not in known or not Path(item["repository"]).is_dir():
|
||||||
return refused("unknown repository: pkgmgr does not list it")
|
return "unknown repository: pkgmgr does not list it"
|
||||||
if UNSAFE.search(item["branch"]) or not item["branch"]:
|
if UNSAFE.search(item["branch"]) or not item["branch"]:
|
||||||
return refused("the branch name holds pattern or option characters; only an exact branch name is deleted")
|
return "the branch name holds pattern or option characters; only an exact branch name is deleted"
|
||||||
code, stdout, stderr = run(self.pkgmgr, plan_argv(item["branch"], False), item["repository"])
|
return None
|
||||||
|
|
||||||
|
def attempt(self, items: list[dict], apply: bool, results: dict[str, dict]) -> tuple[list[dict], int, str, str]:
|
||||||
|
"""Args:
|
||||||
|
items: items of one repository to plan or apply.
|
||||||
|
apply: run the apply instead of the dry run.
|
||||||
|
results: gets, by branch, each item that run names unknown whose
|
||||||
|
ref is gone, as gone "deleted outside this run".
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The items of the last run with its exit code, stdout and stderr;
|
||||||
|
after an "Unknown --branch" for gone branches the run repeats
|
||||||
|
once without them.
|
||||||
|
"""
|
||||||
|
location = items[0]["repository"]
|
||||||
|
code, stdout, stderr = run(self.pkgmgr, plan_argv([item["branch"] for item in items], apply), location, TIMEOUT)
|
||||||
|
named = unknown(stderr) if code != 0 else set()
|
||||||
|
gone = [item for item in items if item["branch"] in named and branch_tip(location, item["branch"]) == ""]
|
||||||
|
if gone:
|
||||||
|
results.update({item["branch"]: {**item, "status": GONE, "reason": DELETED_OUTSIDE, "code": code} for item in gone})
|
||||||
|
items = [item for item in items if item not in gone]
|
||||||
|
if items:
|
||||||
|
code, stdout, stderr = run(self.pkgmgr, plan_argv([item["branch"] for item in items], apply), location, TIMEOUT)
|
||||||
|
return items, code, stdout, stderr
|
||||||
|
|
||||||
|
def repository(self, items: list[dict], default: str, ready: bool) -> list[dict]:
|
||||||
|
"""Args:
|
||||||
|
items: admitted items of one repository, each branch once.
|
||||||
|
default: the repository's default branch.
|
||||||
|
ready: whether pkgmgr holds the GitHub token; without it only a
|
||||||
|
branch signing.token_free() accepts in the dry run is deleted.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Each item with status done, gone, refused or failed and the
|
||||||
|
reason, from one dry run over all of them and one apply over
|
||||||
|
those the dry run plans to delete at the signed tip and worktree,
|
||||||
|
each repeated once without the branches it names unknown and
|
||||||
|
gone; refused items changed nothing.
|
||||||
|
"""
|
||||||
|
results: dict[str, dict] = {}
|
||||||
|
planned, code, stdout, stderr = self.attempt(items, False, results)
|
||||||
plan = plan_items(stdout) if code == 0 else None
|
plan = plan_items(stdout) if code == 0 else None
|
||||||
if plan is None:
|
if plan is None:
|
||||||
return refused(f"the dry run failed ({code}): {stderr.strip()[-500:]}")
|
results.update({item["branch"]: refused(item, f"the dry run failed ({code}): {stderr.strip()[-500:]}") for item in planned})
|
||||||
others = sorted({str(entry.get("branch")) for entry in plan} - {item["branch"]})
|
else:
|
||||||
if others:
|
results.update(screened(planned, plan, default, ready))
|
||||||
return refused(f"the dry run names other branches too, a pattern match: {', '.join(others)[:500]}")
|
chosen = [item for item in planned if item["branch"] not in results]
|
||||||
if len(plan) != 1:
|
if chosen:
|
||||||
return refused(f"the dry run names {len(plan)} items for {item['branch']}")
|
applying, code, stdout, stderr = self.attempt(chosen, True, results)
|
||||||
reason = mismatch(plan[0], item)
|
applied = plan_items(stdout)
|
||||||
if reason:
|
results.update({item["branch"]: applied_result(item, applied, code, stderr) for item in applying})
|
||||||
return refused(reason)
|
return [results[item["branch"]] for item in items]
|
||||||
if not ready and not signing.token_free(plan[0], known[item["repository"]]):
|
|
||||||
return refused(TOKEN_NEEDED)
|
|
||||||
code, stdout, stderr = run(self.pkgmgr, plan_argv(item["branch"], True), item["repository"])
|
|
||||||
applied = plan_items(stdout) or []
|
|
||||||
outcome = next((entry.get("result") for entry in applied if entry.get("branch") == item["branch"]), None)
|
|
||||||
if code == 0 and isinstance(outcome, str) and outcome.split(";")[0] == DONE:
|
|
||||||
return {**item, "status": "done", "reason": "", "code": code, "applied": applied}
|
|
||||||
reason = outcome if isinstance(outcome, str) and outcome else stderr.strip()[-500:] or f"pkgmgr reported no result for {item['branch']}"
|
|
||||||
return {**item, "status": "failed", "reason": reason[:500], "code": code, "applied": applied}
|
|
||||||
|
|
||||||
def job(self, folder: int, path: Path, now: float) -> dict:
|
def job(self, folder: int, path: Path, now: float) -> dict:
|
||||||
try:
|
try:
|
||||||
@@ -205,10 +342,24 @@ class Cleanup:
|
|||||||
if not isinstance(items, list) or not items or not all(map(shaped, items)) or job.get("id") != path.stem:
|
if not isinstance(items, list) or not items or not all(map(shaped, items)) or job.get("id") != path.stem:
|
||||||
return {"id": path.stem, "finished": now, "items": [], "error": "malformed job: refused, nothing deleted"}
|
return {"id": path.stem, "finished": now, "items": [], "error": "malformed job: refused, nothing deleted"}
|
||||||
known, ready = tree_facts(self.tree)
|
known, ready = tree_facts(self.tree)
|
||||||
results = [self.item(item, job.get("proof"), known, ready, now) for item in items]
|
results: list[dict | None] = [None] * len(items)
|
||||||
|
groups: dict[str, list[int]] = {}
|
||||||
|
seen: set[tuple[str, str]] = set()
|
||||||
|
for index, item in enumerate(items):
|
||||||
|
reason = self.admitted(item, job.get("proof"), known, now)
|
||||||
|
if reason is None and (item["repository"], item["branch"]) in seen:
|
||||||
|
reason = "the job lists this branch twice"
|
||||||
|
if reason is not None:
|
||||||
|
results[index] = refused(item, reason)
|
||||||
|
continue
|
||||||
|
seen.add((item["repository"], item["branch"]))
|
||||||
|
groups.setdefault(item["repository"], []).append(index)
|
||||||
|
for location, indexes in groups.items():
|
||||||
|
for index, outcome in zip(indexes, self.repository([items[index] for index in indexes], known[location], ready)):
|
||||||
|
results[index] = outcome
|
||||||
result = {"id": path.stem, "finished": time.time(), "items": results}
|
result = {"id": path.stem, "finished": time.time(), "items": results}
|
||||||
if any(entry["status"] == "done" for entry in results):
|
if any(entry["status"] in (DONE, GONE) for entry in results):
|
||||||
code, _, stderr = run(self.pkgmgr, ["branch", "tree", "--format", "json", "--output", str(self.tree)], str(self.tree.parent))
|
code, _, stderr = run(self.pkgmgr, ["branch", "tree", "--format", "json", "--output", str(self.tree)], str(self.tree.parent), TIMEOUT)
|
||||||
result["tree"] = {"code": code, "error": stderr.strip()[-500:]}
|
result["tree"] = {"code": code, "error": stderr.strip()[-500:]}
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|||||||
Executable
+159
@@ -0,0 +1,159 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Args:
|
||||||
|
--tree: the tree file `pkgmgr branch tree` writes; the plan goes beside
|
||||||
|
it as local-plan.json, in the directory the orchestrator mounts
|
||||||
|
read-only.
|
||||||
|
--pkgmgr: the pkgmgr executable built from a signed tag.
|
||||||
|
--budget: seconds one repository's dry run may take, well below the
|
||||||
|
tree unit's TimeoutStartSec.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Keeps local-plan.json beside the tree file: the dry run of `pkgmgr
|
||||||
|
branch cleanup --local --json` of every repository the tree file says
|
||||||
|
pkgmgr knows, run in it as the cleanup runner runs its dry run
|
||||||
|
(cleanup.run: no repository hook or fsmonitor), named by its real path.
|
||||||
|
A repository whose refs, HEAD, config, reftable and worktree heads
|
||||||
|
(fingerprint) are unchanged since its last attempt is not planned
|
||||||
|
again. Before each dry run the file is rewritten atomically with that
|
||||||
|
repository marked stale ("replan started <time> has not finished") at
|
||||||
|
the attempted fingerprint, and again after it, so a kill keeps every
|
||||||
|
plan written before; a dry run that fails or outlasts --budget keeps
|
||||||
|
the repository's old items, marked stale with the reason and time, and
|
||||||
|
is retried only once its fingerprint changes. The hub offers nothing
|
||||||
|
from a stale repository. Deleting local-plan.json plans every
|
||||||
|
repository again. The exit code is 0 when every dry run of this run
|
||||||
|
printed a plan, else 1.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import datetime
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import cleanup
|
||||||
|
|
||||||
|
LOCAL_PLAN = "local-plan.json"
|
||||||
|
PLAN_ARGV = ["branch", "cleanup", "--local", "--json"]
|
||||||
|
GIT_FILES = ("HEAD", "config", "packed-refs", "reftable/tables.list")
|
||||||
|
WORKTREE_FILES = ("HEAD", "gitdir", "locked", "config.worktree")
|
||||||
|
|
||||||
|
|
||||||
|
def git_dir(repo: Path) -> Path:
|
||||||
|
dot = repo / ".git"
|
||||||
|
if dot.is_file():
|
||||||
|
return (repo / dot.read_text(encoding="utf-8").strip().removeprefix("gitdir: ")).resolve()
|
||||||
|
return dot
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(repo: str) -> str | None:
|
||||||
|
"""Returns:
|
||||||
|
sha256 over name, mtime and size of every file the `--local` plan
|
||||||
|
reads from the repository's git dir: HEAD, config, packed-refs, the
|
||||||
|
reftable table list, every ref, and each worktree's HEAD, gitdir,
|
||||||
|
lock and config; None when the git dir cannot be read.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
root = git_dir(Path(repo))
|
||||||
|
paths = [root / name for name in GIT_FILES]
|
||||||
|
for folder, _, files in os.walk(root / "refs"):
|
||||||
|
paths += [Path(folder, name) for name in files]
|
||||||
|
worktrees = root / "worktrees"
|
||||||
|
if worktrees.is_dir():
|
||||||
|
paths += [entry / name for entry in worktrees.iterdir() for name in WORKTREE_FILES]
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
for path in sorted(paths):
|
||||||
|
try:
|
||||||
|
info = path.stat()
|
||||||
|
except FileNotFoundError:
|
||||||
|
continue
|
||||||
|
digest.update(f"{path.relative_to(root)}\0{info.st_mtime_ns}\0{info.st_size}\n".encode())
|
||||||
|
return digest.hexdigest()
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def known_repos(tree: Path) -> list[str]:
|
||||||
|
data = json.loads(tree.read_text(encoding="utf-8"))
|
||||||
|
return [entry["path"] for entry in data["repos"] if entry.get("known") is True and entry.get("missing") is not True and isinstance(entry.get("path"), str)]
|
||||||
|
|
||||||
|
|
||||||
|
def old_plans(target: Path) -> dict[str, dict]:
|
||||||
|
try:
|
||||||
|
return {entry["repo"]: entry for entry in json.loads(target.read_text(encoding="utf-8"))["repos"] if isinstance(entry, dict) and isinstance(entry.get("repo"), str)}
|
||||||
|
except (OSError, ValueError, KeyError, TypeError, AttributeError):
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def plan(pkgmgr: Path, repo: str, budget: int) -> tuple[dict | None, str]:
|
||||||
|
"""Returns:
|
||||||
|
The repository's {"repo", "default", "merged_into", "warnings",
|
||||||
|
"items"} as pkgmgr prints it, named by its real path, and ""; or
|
||||||
|
None and "timed out" or "failed" when pkgmgr printed no plan of
|
||||||
|
exactly one repository within the budget.
|
||||||
|
"""
|
||||||
|
code, stdout, stderr = cleanup.run(pkgmgr, PLAN_ARGV, repo, budget)
|
||||||
|
if cleanup.plan_items(stdout) is None:
|
||||||
|
return None, "timed out" if code == 127 and "timed out" in stderr else "failed"
|
||||||
|
repos = json.loads(stdout)["repos"]
|
||||||
|
if len(repos) != 1 or not isinstance(repos[0].get("repo"), str):
|
||||||
|
return None, "failed"
|
||||||
|
return {**repos[0], "repo": os.path.realpath(repos[0]["repo"])}, ""
|
||||||
|
|
||||||
|
|
||||||
|
def write(target: Path, data: dict) -> None:
|
||||||
|
with tempfile.NamedTemporaryFile("w", dir=target.parent, prefix=f".{target.name}-", delete=False, encoding="utf-8") as handle:
|
||||||
|
json.dump(data, handle)
|
||||||
|
os.replace(handle.name, target)
|
||||||
|
|
||||||
|
|
||||||
|
def now() -> str:
|
||||||
|
return datetime.datetime.now(datetime.UTC).isoformat(timespec="seconds")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(prog="local_plan")
|
||||||
|
parser.add_argument("--tree", type=Path, required=True)
|
||||||
|
parser.add_argument("--pkgmgr", type=Path, required=True)
|
||||||
|
parser.add_argument("--budget", type=int, required=True)
|
||||||
|
args = parser.parse_args()
|
||||||
|
target = args.tree.with_name(LOCAL_PLAN)
|
||||||
|
try:
|
||||||
|
known = [os.path.realpath(repo) for repo in known_repos(args.tree)]
|
||||||
|
except (OSError, ValueError, KeyError, TypeError, AttributeError) as error:
|
||||||
|
print(f"local_plan: tree file {args.tree} unreadable ({error}), the old plan stays", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
plans = {repo: entry for repo, entry in old_plans(target).items() if repo in known}
|
||||||
|
|
||||||
|
def save() -> None:
|
||||||
|
write(target, {"repos": [plans[repo] for repo in known if repo in plans]})
|
||||||
|
|
||||||
|
failed = []
|
||||||
|
for repo in known:
|
||||||
|
current = fingerprint(repo)
|
||||||
|
if current is not None and plans.get(repo, {}).get("fingerprint") == current:
|
||||||
|
continue
|
||||||
|
previous = plans.get(repo, {"repo": repo, "default": None, "merged_into": [], "warnings": [], "items": []})
|
||||||
|
plans[repo] = {**previous, "fingerprint": current, "stale": f"replan started {now()} has not finished"}
|
||||||
|
save()
|
||||||
|
planned, reason = plan(args.pkgmgr, repo, args.budget)
|
||||||
|
if planned is None:
|
||||||
|
plans[repo] = {**previous, "fingerprint": current, "stale": f"replan {reason} at {now()}"}
|
||||||
|
failed.append(f"{repo}: {reason}")
|
||||||
|
else:
|
||||||
|
plans[repo] = {**planned, "fingerprint": current}
|
||||||
|
save()
|
||||||
|
if not known:
|
||||||
|
save()
|
||||||
|
for line in failed:
|
||||||
|
print(f"local_plan: no --local plan for {line}, the hub offers no delete there until its refs change", file=sys.stderr)
|
||||||
|
return 1 if failed else 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -35,6 +35,7 @@ GATE_ACTIONS = ("commit", "merge", "push", "pr")
|
|||||||
GATE_REPO = re.compile(r"/[^\x00-\x1f\x7f]{0,4095}")
|
GATE_REPO = re.compile(r"/[^\x00-\x1f\x7f]{0,4095}")
|
||||||
GATE_BRANCH = re.compile(r"(?![-/])(?!.*\.\.)(?!.*@\{)(?!.*//)[^\x00-\x20\x7f~^:?*\[\\]{1,255}(?<![/.])")
|
GATE_BRANCH = re.compile(r"(?![-/])(?!.*\.\.)(?!.*@\{)(?!.*//)[^\x00-\x20\x7f~^:?*\[\\]{1,255}(?<![/.])")
|
||||||
TOKEN_FREE_MERGE = "ancestor"
|
TOKEN_FREE_MERGE = "ancestor"
|
||||||
|
LOCAL_DELETE = ["local-branch"]
|
||||||
APPROVAL_KEYS = {"v", "kind", "items", "nonce", "expires"}
|
APPROVAL_KEYS = {"v", "kind", "items", "nonce", "expires"}
|
||||||
RULE_KEYS = {"v", "kind", "id", "kinds", "tool", "pattern", "scope", "expires"}
|
RULE_KEYS = {"v", "kind", "id", "kinds", "tool", "pattern", "scope", "expires"}
|
||||||
TARGET = "agents"
|
TARGET = "agents"
|
||||||
@@ -273,7 +274,8 @@ def names_default(ref: object, default: object) -> bool:
|
|||||||
"""Args:
|
"""Args:
|
||||||
ref: a ref as pkgmgr names it, bare such as main or behind one
|
ref: a ref as pkgmgr names it, bare such as main or behind one
|
||||||
remote such as origin/main, the remote-tracking ref it compares
|
remote such as origin/main, the remote-tracking ref it compares
|
||||||
against when the remote has a HEAD.
|
against when the remote has a HEAD; `branch cleanup --local`
|
||||||
|
names it in full, refs/remotes/origin/main.
|
||||||
default: the repository's default branch as the tree file names it,
|
default: the repository's default branch as the tree file names it,
|
||||||
without the remote (pkgmgr tree.py strips it).
|
without the remote (pkgmgr tree.py strips it).
|
||||||
|
|
||||||
@@ -282,7 +284,7 @@ def names_default(ref: object, default: object) -> bool:
|
|||||||
"""
|
"""
|
||||||
if not isinstance(ref, str) or not isinstance(default, str) or not default:
|
if not isinstance(ref, str) or not isinstance(default, str) or not default:
|
||||||
return False
|
return False
|
||||||
remote, _, name = ref.partition("/")
|
remote, _, name = ref.removeprefix("refs/remotes/").partition("/")
|
||||||
return ref == default or (bool(remote) and name == default)
|
return ref == default or (bool(remote) and name == default)
|
||||||
|
|
||||||
|
|
||||||
@@ -300,6 +302,33 @@ def token_free(entry: dict, default: object) -> bool:
|
|||||||
return entry.get("merged_by") == TOKEN_FREE_MERGE and names_default(entry.get("merged_into"), default) and not entry.get("remotes")
|
return entry.get("merged_by") == TOKEN_FREE_MERGE and names_default(entry.get("merged_into"), default) and not entry.get("remotes")
|
||||||
|
|
||||||
|
|
||||||
|
def local_keeps(entry: dict) -> str:
|
||||||
|
"""Args:
|
||||||
|
entry: one item of `pkgmgr branch cleanup --local --json`, the dry
|
||||||
|
run of what `--apply --local` does to this branch's local tip.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Why that dry run keeps the branch, in pkgmgr's words where it gives
|
||||||
|
them: a state other than merged-clean, its skip reason (for a tip
|
||||||
|
that is no ancestor of the default branch "merged by patch, tip is
|
||||||
|
no ancestor of refs/remotes/origin/main"), a remote branch delete,
|
||||||
|
or a plan without exactly the local-branch delete; "" when it
|
||||||
|
deletes exactly the local branch, the only case the hub offers and
|
||||||
|
the host runs.
|
||||||
|
"""
|
||||||
|
if entry.get("state") != "merged-clean":
|
||||||
|
return f"state is {entry.get('state')}, not merged-clean"
|
||||||
|
if entry.get("skip"):
|
||||||
|
return str(entry["skip"])
|
||||||
|
actions = [str(action) for action in entry.get("actions") or []]
|
||||||
|
remote = [action for action in actions if action.startswith("remote-branch")]
|
||||||
|
if remote:
|
||||||
|
return f"the plan also deletes {', '.join(remote)[:200]}; a signed delete is local only"
|
||||||
|
if actions != LOCAL_DELETE:
|
||||||
|
return f"the dry run plans {', '.join(actions)[:200] or 'no delete'}, not exactly the local branch"
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def approval_payload(hashes: list[str], nonce: str, expires: int) -> dict:
|
def approval_payload(hashes: list[str], nonce: str, expires: int) -> dict:
|
||||||
return {"v": VERSION, "kind": "approve", "items": sorted(set(hashes)), "nonce": nonce, "expires": expires}
|
return {"v": VERSION, "kind": "approve", "items": sorted(set(hashes)), "nonce": nonce, "expires": expires}
|
||||||
|
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ ITEM_LIMIT = 100
|
|||||||
RESULTS_SHOWN = 20
|
RESULTS_SHOWN = 20
|
||||||
TOKEN_ROW = "pkgmgr-github-token"
|
TOKEN_ROW = "pkgmgr-github-token"
|
||||||
TOKEN_COMMAND = "env secret-tool store --label=pkgmgr-github service pkgmgr:github username 'github.com|-' </dev/tty"
|
TOKEN_COMMAND = "env secret-tool store --label=pkgmgr-github service pkgmgr:github username 'github.com|-' </dev/tty"
|
||||||
TOKEN_REASON = "Store the GitHub token in the Secret Service under the key pkgmgr reads; Clean all merged then also deletes remote and squash-merged branches."
|
TOKEN_REASON = "Store the GitHub token in the Secret Service under the key pkgmgr reads; Delete all merged then also deletes the local branch of a merged branch that has a remote branch; never the remote branch."
|
||||||
TOKEN_ADDER = "Operator"
|
TOKEN_ADDER = "Operator"
|
||||||
|
|
||||||
|
|
||||||
@@ -71,8 +71,10 @@ class Cleaning:
|
|||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The signed items, each still a merged-clean branch without a gc
|
The signed items, each still a merged-clean branch without a gc
|
||||||
skip reason at the same tip in the current tree file; without
|
skip reason at the same tip in the current tree file whose
|
||||||
the GitHub token (delete_ready false) each also token-free.
|
`--local` dry run deletes exactly its local branch at that tip
|
||||||
|
(tree.kept_by); without the GitHub token (delete_ready false)
|
||||||
|
each also token-free.
|
||||||
"""
|
"""
|
||||||
if not isinstance(raw, list) or not 1 <= len(raw) <= ITEM_LIMIT:
|
if not isinstance(raw, list) or not 1 <= len(raw) <= ITEM_LIMIT:
|
||||||
raise ToolError(f"items must be a list of 1 to {ITEM_LIMIT} objects")
|
raise ToolError(f"items must be a list of 1 to {ITEM_LIMIT} objects")
|
||||||
@@ -90,7 +92,7 @@ class Cleaning:
|
|||||||
if entry in locked and entry not in known:
|
if entry in locked and entry not in known:
|
||||||
raise ToolError(f"delete blocked: {scan.get('delete_reason') or 'delete_ready is false'}")
|
raise ToolError(f"delete blocked: {scan.get('delete_reason') or 'delete_ready is false'}")
|
||||||
if entry not in known:
|
if entry not in known:
|
||||||
raise ToolError(f"{entry.get('branch')!s:.200} in {entry.get('repository')!s:.200} is no merged-clean branch without skip reason at that tip in the tree file")
|
raise ToolError(f"{entry.get('branch')!s:.200} in {entry.get('repository')!s:.200} is no merged-clean branch without skip reason that the --local dry run deletes at that tip")
|
||||||
if entry in chosen:
|
if entry in chosen:
|
||||||
raise ToolError("items must be distinct")
|
raise ToolError("items must be distinct")
|
||||||
chosen.append(entry)
|
chosen.append(entry)
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ INVALID = "invalid"
|
|||||||
TREE_HEARTBEAT = "tree-heartbeat"
|
TREE_HEARTBEAT = "tree-heartbeat"
|
||||||
GC_SKILL = "kvb-git-gc"
|
GC_SKILL = "kvb-git-gc"
|
||||||
GC_COMPOSE_DOWN = ("docker", "compose", "-p", "{project}", "down", "-v", "--remove-orphans")
|
GC_COMPOSE_DOWN = ("docker", "compose", "-p", "{project}", "down", "-v", "--remove-orphans")
|
||||||
|
LOCAL_PLAN = "local-plan.json"
|
||||||
|
|
||||||
|
|
||||||
def flow(item: dict) -> dict | None:
|
def flow(item: dict) -> dict | None:
|
||||||
@@ -72,15 +73,64 @@ def held_by(worktree: str) -> str:
|
|||||||
return f"held by worktree {worktree}: pkgmgr branch cleanup skips it, use branch gc ({GC_SKILL}, pkgmgr gc)"
|
return f"held by worktree {worktree}: pkgmgr branch cleanup skips it, use branch gc ({GC_SKILL}, pkgmgr gc)"
|
||||||
|
|
||||||
|
|
||||||
def dry_run(item: dict) -> list[str]:
|
def kept_by(planned: dict | None, tip: str, upstream_of: str, stale: str) -> str:
|
||||||
"""Returns:
|
"""Args:
|
||||||
|
planned: this branch's item of the `pkgmgr branch cleanup --local`
|
||||||
|
dry run beside the tree file, None when it has none.
|
||||||
|
tip: the branch's local tip in the tree file.
|
||||||
|
upstream_of: a branch of the same dry run whose local upstream this
|
||||||
|
branch is, "" for none; pkgmgr's _config_keeps keeps it when the
|
||||||
|
host's `--branch` dry run chooses it alone.
|
||||||
|
stale: why the repository's dry run is stale, "" when it is fresh.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Why the hub offers no delete: a stale dry run, or from that dry run
|
||||||
|
what keeps the branch (signing.local_keeps, the predicate the host
|
||||||
|
runner refuses by, then the local upstream), or that it has not
|
||||||
|
planned this tip yet; "" when it deletes exactly the local branch at
|
||||||
|
this tip.
|
||||||
|
"""
|
||||||
|
if stale:
|
||||||
|
return f"pkgmgr branch cleanup --local plan stale: {stale}; use branch gc ({GC_SKILL}, pkgmgr gc)"
|
||||||
|
if planned is None or planned.get("local_sha") != tip:
|
||||||
|
return f"no pkgmgr branch cleanup --local dry run of {tip[:12]} yet: the tree timer writes {LOCAL_PLAN} within a minute"
|
||||||
|
keeps = signing.local_keeps(planned) or (f"kept: local upstream of {upstream_of}" if upstream_of else "")
|
||||||
|
return f"pkgmgr branch cleanup --local keeps it: {keeps}; use branch gc ({GC_SKILL}, pkgmgr gc)" if keeps else ""
|
||||||
|
|
||||||
|
|
||||||
|
def local_upstreams(items: list[dict]) -> dict[str, str]:
|
||||||
|
"""Args:
|
||||||
|
items: one repository's items of the `--local` dry run, each with
|
||||||
|
its `%(upstream:short)` as upstream.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Each branch that is the local upstream of another branch there, with
|
||||||
|
the first such branch by name, as pkgmgr's _config_keeps names it.
|
||||||
|
"""
|
||||||
|
names = {item["branch"] for item in items}
|
||||||
|
found: dict[str, str] = {}
|
||||||
|
for item in sorted(items, key=lambda entry: entry["branch"]):
|
||||||
|
upstream = str(item.get("upstream") or "")
|
||||||
|
parent = upstream if upstream in names else upstream.removeprefix("heads/")
|
||||||
|
if parent in names and parent != item["branch"]:
|
||||||
|
found.setdefault(parent, item["branch"])
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
def dry_run(item: dict, kept: str) -> list[str]:
|
||||||
|
"""Args:
|
||||||
|
item: one tree item.
|
||||||
|
kept: kept_by() for it, "" when the hub offers its delete.
|
||||||
|
|
||||||
|
Returns:
|
||||||
What the runner's `pkgmgr branch cleanup` does for one tree item,
|
What the runner's `pkgmgr branch cleanup` does for one tree item,
|
||||||
read from the gc plan the tree file carries: it deletes the local
|
read from the gc plan the tree file carries: it deletes the local
|
||||||
branch and never touches a worktree, Compose stack or volume; a
|
branch and never touches a worktree, Compose stack or volume; a
|
||||||
remote branch stays, the runner refuses a plan that deletes one. A
|
remote branch stays, the runner refuses a plan that deletes one. A
|
||||||
branch held by a worktree it skips; its lines name what branch gc
|
branch held by a worktree it skips, and one the `--local` dry run
|
||||||
would do instead, its `docker compose down -v` deleting the stack's
|
keeps or has not planned it does not delete; their lines name what
|
||||||
volumes.
|
branch gc would do instead, its `docker compose down -v` deleting
|
||||||
|
the stack's volumes.
|
||||||
"""
|
"""
|
||||||
tip = (item["local_sha"] or item["head"] or "")[:12]
|
tip = (item["local_sha"] or item["head"] or "")[:12]
|
||||||
merged = f", merged by {item['merged_by']}" if item["merged_by"] else ""
|
merged = f", merged by {item['merged_by']}" if item["merged_by"] else ""
|
||||||
@@ -91,6 +141,10 @@ def dry_run(item: dict) -> list[str]:
|
|||||||
lines += [f"branch gc would run {' '.join(GC_COMPOSE_DOWN).format(project=name)}, which deletes the volumes of {name}" for name in item["compose_projects"]]
|
lines += [f"branch gc would run {' '.join(GC_COMPOSE_DOWN).format(project=name)}, which deletes the volumes of {name}" for name in item["compose_projects"]]
|
||||||
lines += [f"branch gc would remove the image {name}" for name in item["images"]]
|
lines += [f"branch gc would remove the image {name}" for name in item["images"]]
|
||||||
return lines
|
return lines
|
||||||
|
if kept:
|
||||||
|
lines.append(kept)
|
||||||
|
lines += [f"branch gc would delete the local branch {item['branch']}" if action == "local-branch" else f"branch gc would delete {action}" for action in item["actions"]]
|
||||||
|
return lines
|
||||||
for action in item["actions"]:
|
for action in item["actions"]:
|
||||||
if action == "local-branch":
|
if action == "local-branch":
|
||||||
lines.append(f"deletes the local branch {item['branch']}")
|
lines.append(f"deletes the local branch {item['branch']}")
|
||||||
@@ -101,14 +155,25 @@ def dry_run(item: dict) -> list[str]:
|
|||||||
return lines
|
return lines
|
||||||
|
|
||||||
|
|
||||||
def branch(item: dict, path: str, default: object) -> dict:
|
def branch(item: dict, path: str, default: object, planned: dict | None, upstream_of: str, stale: str) -> dict:
|
||||||
|
"""Args:
|
||||||
|
item: one tree item with a branch.
|
||||||
|
path: its repository's path.
|
||||||
|
default: the repository's default branch.
|
||||||
|
planned: the branch's item of the `--local` dry run, None without.
|
||||||
|
upstream_of: as kept_by() takes it.
|
||||||
|
stale: as kept_by() takes it.
|
||||||
|
"""
|
||||||
remote = next(iter(item["remotes"].values()), "")
|
remote = next(iter(item["remotes"].values()), "")
|
||||||
head = item["local_sha"] or item["head"] or remote
|
head = item["local_sha"] or item["head"] or remote
|
||||||
held = held_by(item["worktree"]) if item["worktree"] and item["state"] == "merged-clean" else ""
|
held = held_by(item["worktree"]) if item["worktree"] and item["state"] == "merged-clean" else ""
|
||||||
open_to_delete = item["state"] == "merged-clean" and not item["skip"] and bool(item["local_sha"]) and not item["worktree"]
|
candidate = item["state"] == "merged-clean" and bool(item["local_sha"]) and not item["worktree"]
|
||||||
|
kept = kept_by(planned, item["local_sha"], upstream_of, stale) if candidate else ""
|
||||||
|
open_to_delete = candidate and not item["skip"] and not kept
|
||||||
return {
|
return {
|
||||||
"held": held,
|
"held": held,
|
||||||
"token_free": signing.token_free(item, default),
|
"kept": kept,
|
||||||
|
"token_free": signing.token_free(planned, default) if candidate and planned is not None else False,
|
||||||
"name": item["branch"],
|
"name": item["branch"],
|
||||||
"head": head,
|
"head": head,
|
||||||
"upstream": item["upstream"] or "",
|
"upstream": item["upstream"] or "",
|
||||||
@@ -120,22 +185,45 @@ def branch(item: dict, path: str, default: object) -> dict:
|
|||||||
"merged_into": item["merged_into"],
|
"merged_into": item["merged_into"],
|
||||||
"skip": item["skip"],
|
"skip": item["skip"],
|
||||||
"reason": item["reason"],
|
"reason": item["reason"],
|
||||||
"plan": dry_run(item),
|
"plan": dry_run(item, kept),
|
||||||
"cleanup": signing.cleanup_item(path, item["branch"], item["worktree"] or "", item["local_sha"]) if open_to_delete else None,
|
"cleanup": signing.cleanup_item(path, item["branch"], item["worktree"] or "", item["local_sha"]) if open_to_delete else None,
|
||||||
"worktrees": [worktree(item)] if item["worktree"] else [],
|
"worktrees": [worktree(item)] if item["worktree"] else [],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def repo(entry: dict) -> dict:
|
def repo(entry: dict, local: dict[tuple[str, str], tuple[dict, str]], stale: dict[str, str]) -> dict:
|
||||||
items = entry["items"]
|
items = entry["items"]
|
||||||
return {
|
return {
|
||||||
**{key: value for key, value in entry.items() if key != "items"},
|
**{key: value for key, value in entry.items() if key != "items"},
|
||||||
"name": "/".join(Path(entry["path"]).parts[-LABEL_PARTS:]),
|
"name": "/".join(Path(entry["path"]).parts[-LABEL_PARTS:]),
|
||||||
"branches": [branch(item, entry["path"], entry.get("default")) for item in items if item["branch"]],
|
"branches": [branch(item, entry["path"], entry.get("default"), *local.get((entry["path"], item["branch"]), (None, "")), stale.get(entry["path"], "")) for item in items if item["branch"]],
|
||||||
"detached": [worktree(item) for item in items if item["detached"]],
|
"detached": [worktree(item) for item in items if item["detached"]],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def local_plan(path: Path) -> tuple[dict[tuple[str, str], tuple[dict, str]], dict[str, str]]:
|
||||||
|
"""Returns:
|
||||||
|
Every item of the `pkgmgr branch cleanup --local` dry run that
|
||||||
|
client/local_plan.py writes at path, keyed by repository path and
|
||||||
|
branch, with the branch whose local upstream it is (local_upstreams,
|
||||||
|
"" for none); and each repository whose dry run is stale, with why.
|
||||||
|
Both are empty when the file is missing or unreadable, so the hub
|
||||||
|
offers no delete.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
found, stale = {}, {}
|
||||||
|
for repository in data["repos"]:
|
||||||
|
items = [item for item in repository["items"] if isinstance(item, dict) and isinstance(item.get("branch"), str)]
|
||||||
|
upstreams = local_upstreams(items)
|
||||||
|
found.update({(repository["repo"], item["branch"]): (item, upstreams.get(item["branch"], "")) for item in items})
|
||||||
|
if repository.get("stale"):
|
||||||
|
stale[repository["repo"]] = str(repository["stale"])[:200]
|
||||||
|
return found, stale
|
||||||
|
except (OSError, ValueError, KeyError, TypeError, AttributeError):
|
||||||
|
return {}, {}
|
||||||
|
|
||||||
|
|
||||||
def offered(scan: dict, ready: bool) -> list[dict]:
|
def offered(scan: dict, ready: bool) -> list[dict]:
|
||||||
"""Args:
|
"""Args:
|
||||||
scan: the tree.
|
scan: the tree.
|
||||||
@@ -151,8 +239,9 @@ def offered(scan: dict, ready: bool) -> list[dict]:
|
|||||||
def read_tree(path: Path) -> dict:
|
def read_tree(path: Path) -> dict:
|
||||||
try:
|
try:
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
local, stale = local_plan(path.with_name(LOCAL_PLAN))
|
||||||
return {
|
return {
|
||||||
"repos": sorted((repo(entry) for entry in data["repos"]), key=lambda entry: entry["name"]),
|
"repos": sorted((repo(entry, local, stale) for entry in data["repos"]), key=lambda entry: entry["name"]),
|
||||||
"scanned": iso_epoch(data["generated_at"]),
|
"scanned": iso_epoch(data["generated_at"]),
|
||||||
"delete_ready": data["delete_ready"],
|
"delete_ready": data["delete_ready"],
|
||||||
"delete_reason": data["delete_reason"],
|
"delete_reason": data["delete_reason"],
|
||||||
@@ -204,7 +293,7 @@ class Tree:
|
|||||||
heartbeat: Path
|
heartbeat: Path
|
||||||
lock: threading.Lock
|
lock: threading.Lock
|
||||||
cached: dict | None
|
cached: dict | None
|
||||||
stamp: tuple[int, int] | None
|
stamp: tuple[tuple[int, int], tuple[int, int] | None] | None
|
||||||
|
|
||||||
def beat(self) -> None:
|
def beat(self) -> None:
|
||||||
self.heartbeat.touch()
|
self.heartbeat.touch()
|
||||||
@@ -218,7 +307,12 @@ class Tree:
|
|||||||
f"no pkgmgr tree file at {self.file}: {error.strerror}. The tree timer runs `pkgmgr branch tree`, which the installed pkgmgr "
|
f"no pkgmgr tree file at {self.file}: {error.strerror}. The tree timer runs `pkgmgr branch tree`, which the installed pkgmgr "
|
||||||
"lacks until a release ships it: release pkgmgr and install it, then the timer writes the file within a minute."
|
"lacks until a release ships it: release pkgmgr and install it, then the timer writes the file within a minute."
|
||||||
)}
|
)}
|
||||||
stamp = (info.st_mtime_ns, info.st_size)
|
try:
|
||||||
|
planned = self.file.with_name(LOCAL_PLAN).stat()
|
||||||
|
plan_stamp = (planned.st_mtime_ns, planned.st_size)
|
||||||
|
except OSError:
|
||||||
|
plan_stamp = None
|
||||||
|
stamp = ((info.st_mtime_ns, info.st_size), plan_stamp)
|
||||||
if self.cached is None or stamp != self.stamp:
|
if self.cached is None or stamp != self.stamp:
|
||||||
self.cached, self.stamp = read_tree(self.file), stamp
|
self.cached, self.stamp = read_tree(self.file), stamp
|
||||||
return self.cached
|
return self.cached
|
||||||
@@ -1,11 +1,12 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=Agent Orchestrator tree: pkgmgr branch tree into the file the orchestrator reads, while its page polls
|
Description=Agent Orchestrator tree: pkgmgr branch tree and the pkgmgr branch cleanup --local dry run into the files the orchestrator reads, while its page polls
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
Type=oneshot
|
Type=oneshot
|
||||||
Environment=PATH=/usr/bin:/bin
|
Environment=PATH=/usr/bin:/bin
|
||||||
ExecCondition=/bin/sh -c 'test -n "$(find %h/.local/state/kvb-orchestrator/tree-heartbeat -mmin -5 2>/dev/null)"'
|
ExecCondition=/bin/sh -c 'test -n "$(find %h/.local/state/kvb-orchestrator/tree-heartbeat -mmin -5 2>/dev/null)"'
|
||||||
ExecStart=%h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json
|
ExecStart=%h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json
|
||||||
|
ExecStart=/usr/bin/env python3 %h/.local/share/kvb-orchestrator/client/local_plan.py --tree %h/.local/state/pkgmgr/tree/branch-tree.json --pkgmgr %h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr --budget 240
|
||||||
Nice=10
|
Nice=10
|
||||||
IOSchedulingClass=idle
|
IOSchedulingClass=idle
|
||||||
CPUQuota=50%
|
CPUQuota=50%
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
const CLEANUP_POLL = 2000;
|
const CLEANUP_POLL = 2000;
|
||||||
const CLEANUP_PATIENCE = 900000;
|
const CLEANUP_PATIENCE = 900000;
|
||||||
const RESULT_LABELS = {done: ["deleted", "ok"], refused: ["refused", "bad"], failed: ["failed", "bad"]};
|
const RESULT_LABELS = {done: ["deleted", "ok"], gone: ["gone", "warn"], refused: ["refused", "bad"], failed: ["failed", "bad"]};
|
||||||
|
|
||||||
function planOf(item) {
|
function planOf(item) {
|
||||||
const repo = (tree.data.repos || []).find((entry) => entry.path === item.repository);
|
const repo = (tree.data.repos || []).find((entry) => entry.path === item.repository);
|
||||||
@@ -8,16 +8,18 @@
|
|||||||
return {repo, entry};
|
return {repo, entry};
|
||||||
}
|
}
|
||||||
|
|
||||||
function deleteDialog(items, locked = [], held = []) {
|
function deleteDialog(items, locked = [], held = [], kept = []) {
|
||||||
if (!items.length) {
|
if (!items.length) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const rows = [...items, ...locked, ...held].map((item) => {
|
const rows = [...items, ...locked, ...held, ...kept].map((item) => {
|
||||||
const {repo, entry} = planOf(item);
|
const {repo, entry} = planOf(item);
|
||||||
const waits = locked.includes(item);
|
const waits = locked.includes(item);
|
||||||
const kept = held.includes(item);
|
const holds = held.includes(item);
|
||||||
return h("li", {class: waits ? "dry-run locked" : kept ? "dry-run held" : "dry-run", "data-token": waits ? "needed" : null, "data-held": kept && entry ? entry.held : null},
|
const keeps = kept.includes(item);
|
||||||
h("div", {class: "row"}, h("strong", {}, revealed(repo ? repo.name : item.repository)), h("code", {}, revealed(item.branch)), waits ? chip("bi-key", "token needed", "warn") : kept ? chip("bi-folder-symlink", "held by a worktree: not deleted here", "neutral") : null),
|
return h("li", {class: waits ? "dry-run locked" : holds ? "dry-run held" : keeps ? "dry-run kept" : "dry-run", "data-token": waits ? "needed" : null, "data-held": holds && entry ? entry.held : null, "data-kept": keeps && entry ? entry.kept : null},
|
||||||
|
h("div", {class: "row"}, h("strong", {}, revealed(repo ? repo.name : item.repository)), h("code", {}, revealed(item.branch)),
|
||||||
|
waits ? chip("bi-key", "token needed", "warn") : holds ? chip("bi-folder-symlink", "held by a worktree: not deleted here", "neutral") : keeps ? chip("bi-sign-merge-left", "kept by the --local dry run: not deleted here", "neutral") : null),
|
||||||
h("pre", {}, (entry ? entry.plan : [`${item.branch} @ ${item.tip.slice(0, 12)}`]).map(revealed).join("\n")));
|
h("pre", {}, (entry ? entry.plan : [`${item.branch} @ ${item.tip.slice(0, 12)}`]).map(revealed).join("\n")));
|
||||||
});
|
});
|
||||||
const outcome = h("div", {class: "mini", "aria-live": "polite"});
|
const outcome = h("div", {class: "mini", "aria-live": "polite"});
|
||||||
@@ -29,7 +31,7 @@
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
openSheet(items.length > 1 ? `Delete ${items.length} merged branches` : `Delete ${items[0].branch}`, [
|
openSheet(items.length > 1 ? `Delete ${items.length} merged branches` : `Delete ${items[0].branch}`, [
|
||||||
h("p", {class: "muted"}, "Dry run of pkgmgr branch cleanup for exactly these entries: it deletes the local branch and never touches a worktree, Compose stack or volume. No remote branch: the host refuses an entry whose plan deletes one. A branch held by a worktree is not deleted here: branch gc (kvb-git-gc, pkgmgr gc) removes it, and its docker compose down -v deletes the stack's volumes. The host checks your signature and refuses every entry whose state has changed since."),
|
h("p", {class: "muted"}, "Dry run of pkgmgr branch cleanup for exactly these entries: it deletes the local branch and never touches a worktree, Compose stack or volume. No remote branch: the host refuses an entry whose plan deletes one. A branch held by a worktree is not deleted here: branch gc (kvb-git-gc, pkgmgr gc) removes it, and its docker compose down -v deletes the stack's volumes. Neither is a branch the pkgmgr branch cleanup --local dry run keeps, such as one merged by pull request, squash or patch whose local tip is no ancestor of the default branch: branch gc removes it. The host checks your signature and refuses every entry whose state has changed since."),
|
||||||
h("ul", {class: "mini", "aria-label": "Dry run"}, rows),
|
h("ul", {class: "mini", "aria-label": "Dry run"}, rows),
|
||||||
outcome,
|
outcome,
|
||||||
h("div", {class: "actions"}, h("span", {class: "spacer"}), btn("Cancel", "btn ghost", closeSheet), confirm),
|
h("div", {class: "actions"}, h("span", {class: "spacer"}), btn("Cancel", "btn ghost", closeSheet), confirm),
|
||||||
|
|||||||
@@ -27,12 +27,15 @@
|
|||||||
|
|
||||||
const FORGE_UNAVAILABLE = "forge unavailable";
|
const FORGE_UNAVAILABLE = "forge unavailable";
|
||||||
const deletableNow = (entry) => Boolean(entry.cleanup) && (tree.data.delete_ready === true || entry.token_free === true);
|
const deletableNow = (entry) => Boolean(entry.cleanup) && (tree.data.delete_ready === true || entry.token_free === true);
|
||||||
const tokenLocked = (entry) => tree.data.delete_ready !== true && entry.state === "merged-clean" && !entry.held && (entry.cleanup ? entry.token_free !== true : entry.skip === FORGE_UNAVAILABLE);
|
const tokenLocked = (entry) => tree.data.delete_ready !== true && entry.state === "merged-clean" && !entry.held && !entry.kept && (entry.cleanup ? entry.token_free !== true : entry.skip === FORGE_UNAVAILABLE);
|
||||||
|
|
||||||
function deleteButton(entry) {
|
function deleteButton(entry) {
|
||||||
if (entry.held) {
|
if (entry.held) {
|
||||||
return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));
|
return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));
|
||||||
}
|
}
|
||||||
|
if (entry.kept) {
|
||||||
|
return h("span", {class: "hint kept", title: entry.kept}, chip("bi-sign-merge-left", entry.kept, "neutral"));
|
||||||
|
}
|
||||||
if (!entry.cleanup) {
|
if (!entry.cleanup) {
|
||||||
return tokenLocked(entry) ? chip("bi-key", "token needed", "warn") : null;
|
return tokenLocked(entry) ? chip("bi-key", "token needed", "warn") : null;
|
||||||
}
|
}
|
||||||
@@ -53,14 +56,17 @@
|
|||||||
const items = branches.filter(deletableNow).map((entry) => entry.cleanup);
|
const items = branches.filter(deletableNow).map((entry) => entry.cleanup);
|
||||||
const waiting = branches.filter(tokenLocked);
|
const waiting = branches.filter(tokenLocked);
|
||||||
const locked = waiting.filter((entry) => entry.cleanup).map((entry) => entry.cleanup);
|
const locked = waiting.filter((entry) => entry.cleanup).map((entry) => entry.cleanup);
|
||||||
const held = (tree.data.repos || []).flatMap((repo) => repo.branches.filter((entry) => entry.held).map((entry) => ({repository: repo.path, branch: entry.name, tip: entry.head})));
|
const preview = (key) => (tree.data.repos || []).flatMap((repo) => repo.branches.filter((entry) => entry[key]).map((entry) => ({repository: repo.path, branch: entry.name, tip: entry.head})));
|
||||||
|
const held = preview("held");
|
||||||
|
const kept = preview("kept");
|
||||||
const ready = items.length > 0;
|
const ready = items.length > 0;
|
||||||
const hint = !ready && tree.data.delete_ready !== true ? `Delete all merged blocked: ${tree.data.delete_reason || "delete_ready is false"}` : items.length ? `Delete all merged: ${items.length} branches after dry run and signature` : "Delete all merged: no merged branch without a skip reason";
|
const hint = !ready && tree.data.delete_ready !== true ? `Delete all merged blocked: ${tree.data.delete_reason || "delete_ready is false"}` : items.length ? `Delete all merged: ${items.length} branches after dry run and signature` : "Delete all merged: no merged branch without a skip reason";
|
||||||
const button = iconBtn("bi-trash3-fill", "Delete all merged", "btn small danger", () => deleteDialog(items, locked, held), hint);
|
const button = iconBtn("bi-trash3-fill", "Delete all merged", "btn small danger", () => deleteDialog(items, locked, held, kept), hint);
|
||||||
button.disabled = !ready;
|
button.disabled = !ready;
|
||||||
return h("span", {class: "hint", title: hint}, button, items.length ? counter(items.length) : null,
|
return h("span", {class: "hint", title: hint}, button, items.length ? counter(items.length) : null,
|
||||||
waiting.length ? chip("bi-key", `${waiting.length} token needed`, "warn", waiting.length) : null,
|
waiting.length ? chip("bi-key", `${waiting.length} token needed`, "warn", waiting.length) : null,
|
||||||
held.length ? chip("bi-folder-symlink", `${held.length} held by a worktree: use branch gc`, "neutral", held.length) : null,
|
held.length ? chip("bi-folder-symlink", `${held.length} held by a worktree: use branch gc`, "neutral", held.length) : null,
|
||||||
|
kept.length ? chip("bi-sign-merge-left", `${kept.length} kept by the --local dry run: see each dry run`, "neutral", kept.length) : null,
|
||||||
tree.data.delete_ready !== true ? iconBtn("bi-key-fill", "Store token", "btn small soft", storeToken, "Store token: add the operator row that stores the GitHub token in the Secret Service") : null);
|
tree.data.delete_ready !== true ? iconBtn("bi-key-fill", "Store token", "btn small soft", storeToken, "Store token: add the operator row that stores the GitHub token in the Secret Service") : null);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -92,8 +98,10 @@
|
|||||||
return found.length ? found[found.length - 1] : null;
|
return found.length ? found[found.length - 1] : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const RESULT_CHIPS = {done: ["bi-check2-all", "deleted", "ok"], gone: ["bi-question-octagon", "gone", "warn"], refused: ["bi-x-octagon", "refused", "bad"], failed: ["bi-exclamation-triangle", "failed", "bad"]};
|
||||||
|
|
||||||
function resultChip(result) {
|
function resultChip(result) {
|
||||||
const [name, label, tone] = result.status === "done" ? ["bi-check2-all", "deleted", "ok"] : result.status === "refused" ? ["bi-x-octagon", "refused", "bad"] : ["bi-exclamation-triangle", "failed", "bad"];
|
const [name, label, tone] = RESULT_CHIPS[result.status] || RESULT_CHIPS.failed;
|
||||||
return chip(name, `${label}${result.reason ? `: ${result.reason}` : ""}`, tone);
|
return chip(name, `${label}${result.reason ? `: ${result.reason}` : ""}`, tone);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,144 @@
|
|||||||
|
{
|
||||||
|
"repos": [
|
||||||
|
{
|
||||||
|
"repo": "@REPO@",
|
||||||
|
"default": "refs/remotes/origin/main",
|
||||||
|
"merged_into": [],
|
||||||
|
"warnings": [],
|
||||||
|
"items": [
|
||||||
|
{
|
||||||
|
"branch": "main",
|
||||||
|
"worktree": "@REPO@",
|
||||||
|
"head": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f",
|
||||||
|
"detached": false,
|
||||||
|
"locked": null,
|
||||||
|
"worktree_state": "clean",
|
||||||
|
"staged": 0,
|
||||||
|
"unstaged": 0,
|
||||||
|
"untracked": 0,
|
||||||
|
"upstream": null,
|
||||||
|
"ahead": null,
|
||||||
|
"behind": null,
|
||||||
|
"gone": false,
|
||||||
|
"local_sha": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f",
|
||||||
|
"remotes": {
|
||||||
|
"origin": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f"
|
||||||
|
},
|
||||||
|
"compose_projects": [],
|
||||||
|
"images": [],
|
||||||
|
"state": "protected",
|
||||||
|
"merged": false,
|
||||||
|
"dirty": 0,
|
||||||
|
"merged_by": null,
|
||||||
|
"merged_into": null,
|
||||||
|
"unpushed": null,
|
||||||
|
"last_activity": null,
|
||||||
|
"size_bytes": null,
|
||||||
|
"skip": null,
|
||||||
|
"actions": [],
|
||||||
|
"reason": "default or protected branch",
|
||||||
|
"result": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"branch": "side",
|
||||||
|
"worktree": null,
|
||||||
|
"head": null,
|
||||||
|
"detached": false,
|
||||||
|
"locked": null,
|
||||||
|
"worktree_state": null,
|
||||||
|
"staged": null,
|
||||||
|
"unstaged": null,
|
||||||
|
"untracked": null,
|
||||||
|
"upstream": null,
|
||||||
|
"ahead": null,
|
||||||
|
"behind": null,
|
||||||
|
"gone": false,
|
||||||
|
"local_sha": "cf287347ef836eeebeab3c87cdb149d99a3cee91",
|
||||||
|
"remotes": {},
|
||||||
|
"compose_projects": [],
|
||||||
|
"images": [],
|
||||||
|
"state": "merged-clean",
|
||||||
|
"merged": true,
|
||||||
|
"dirty": 0,
|
||||||
|
"merged_by": "ancestor",
|
||||||
|
"merged_into": "refs/remotes/origin/main",
|
||||||
|
"unpushed": null,
|
||||||
|
"last_activity": null,
|
||||||
|
"size_bytes": null,
|
||||||
|
"skip": null,
|
||||||
|
"actions": [
|
||||||
|
"local-branch"
|
||||||
|
],
|
||||||
|
"reason": "merged by ancestor",
|
||||||
|
"result": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"branch": "squashed",
|
||||||
|
"worktree": null,
|
||||||
|
"head": null,
|
||||||
|
"detached": false,
|
||||||
|
"locked": null,
|
||||||
|
"worktree_state": null,
|
||||||
|
"staged": null,
|
||||||
|
"unstaged": null,
|
||||||
|
"untracked": null,
|
||||||
|
"upstream": null,
|
||||||
|
"ahead": null,
|
||||||
|
"behind": null,
|
||||||
|
"gone": false,
|
||||||
|
"local_sha": "73e7e81781ea1bf4c3fb0b1a9aa99e7809529e7d",
|
||||||
|
"remotes": {},
|
||||||
|
"compose_projects": [],
|
||||||
|
"images": [],
|
||||||
|
"state": "merged-clean",
|
||||||
|
"merged": true,
|
||||||
|
"dirty": 0,
|
||||||
|
"merged_by": "patch",
|
||||||
|
"merged_into": "refs/remotes/origin/main",
|
||||||
|
"unpushed": null,
|
||||||
|
"last_activity": null,
|
||||||
|
"size_bytes": null,
|
||||||
|
"skip": "merged by patch, tip is no ancestor of refs/remotes/origin/main",
|
||||||
|
"actions": [],
|
||||||
|
"reason": "merged by patch",
|
||||||
|
"result": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"branch": "topic",
|
||||||
|
"worktree": null,
|
||||||
|
"head": null,
|
||||||
|
"detached": false,
|
||||||
|
"locked": null,
|
||||||
|
"worktree_state": null,
|
||||||
|
"staged": null,
|
||||||
|
"unstaged": null,
|
||||||
|
"untracked": null,
|
||||||
|
"upstream": null,
|
||||||
|
"ahead": null,
|
||||||
|
"behind": null,
|
||||||
|
"gone": false,
|
||||||
|
"local_sha": "ab372c4ea8b3ee5d062f1f5ea43c8154bf70b62a",
|
||||||
|
"remotes": {
|
||||||
|
"origin": "99dce520c1c6076008c6178cba98b360d41dbd0d"
|
||||||
|
},
|
||||||
|
"compose_projects": [],
|
||||||
|
"images": [],
|
||||||
|
"state": "merged-clean",
|
||||||
|
"merged": true,
|
||||||
|
"dirty": 0,
|
||||||
|
"merged_by": "patch",
|
||||||
|
"merged_into": "refs/remotes/origin/main",
|
||||||
|
"unpushed": null,
|
||||||
|
"last_activity": null,
|
||||||
|
"size_bytes": null,
|
||||||
|
"skip": null,
|
||||||
|
"actions": [
|
||||||
|
"local-branch"
|
||||||
|
],
|
||||||
|
"reason": "merged by patch",
|
||||||
|
"result": null
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -78,7 +78,7 @@
|
|||||||
"demo:feature-done"
|
"demo:feature-done"
|
||||||
],
|
],
|
||||||
"state": "merged-clean",
|
"state": "merged-clean",
|
||||||
"merged_by": "#12",
|
"merged_by": "pr#12",
|
||||||
"merged_into": "main",
|
"merged_into": "main",
|
||||||
"size_bytes": 4096,
|
"size_bytes": 4096,
|
||||||
"skip": null,
|
"skip": null,
|
||||||
@@ -87,7 +87,7 @@
|
|||||||
"local-branch",
|
"local-branch",
|
||||||
"remote-branch"
|
"remote-branch"
|
||||||
],
|
],
|
||||||
"reason": "merged into main by #12",
|
"reason": "merged by pr#12",
|
||||||
"result": null,
|
"result": null,
|
||||||
"flow": null
|
"flow": null
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -297,7 +297,7 @@ class TestOrchestratorGaps(HubCase):
|
|||||||
self.assertNotIn("error", icons, icons)
|
self.assertNotIn("error", icons, icons)
|
||||||
chips = icons["chips"]
|
chips = icons["chips"]
|
||||||
expected = {
|
expected = {
|
||||||
"merged by #12 into main": "#bi-sign-merge-left", "clean": "#bi-check-circle", "not committed: 0 staged, 0 changed, 1 new": "#bi-pencil-square", "1 commits unpushed": "#bi-cloud-upload",
|
"merged by pr#12 into main": "#bi-sign-merge-left", "clean": "#bi-check-circle", "not committed: 0 staged, 0 changed, 1 new": "#bi-pencil-square", "1 commits unpushed": "#bi-cloud-upload",
|
||||||
"upstream gone": "#bi-cloud-slash", f"locked: {LOCKED}": "#bi-lock-fill", "stale": "#bi-hourglass-bottom", "protected": "#bi-shield-lock",
|
"upstream gone": "#bi-cloud-slash", f"locked: {LOCKED}": "#bi-lock-fill", "stale": "#bi-hourglass-bottom", "protected": "#bi-shield-lock",
|
||||||
}
|
}
|
||||||
for label, href in expected.items():
|
for label, href in expected.items():
|
||||||
|
|||||||
@@ -384,6 +384,45 @@ AFTER = """(async () => {@HELPERS@
|
|||||||
const past = await until(() => all("#main details.repo").find((details) => details.textContent.includes("Recently deleted")), 15000);
|
const past = await until(() => all("#main details.repo").find((details) => details.textContent.includes("Recently deleted")), 15000);
|
||||||
return {past: past ? [past.querySelector("[data-result]").dataset.result, name(past.querySelector("span.state"))] : null, branches: all("#main details.repo").filter((details) => details !== past).flatMap((details) => all(".branch-name strong", details)).map(text)};
|
return {past: past ? [past.querySelector("[data-result]").dataset.result, name(past.querySelector("span.state"))] : null, branches: all("#main details.repo").filter((details) => details !== past).flatMap((details) => all(".branch-name strong", details)).map(text)};
|
||||||
})()""".replace("@HELPERS@", HELPERS)
|
})()""".replace("@HELPERS@", HELPERS)
|
||||||
|
KEPT_TREE = """(async () => {@HELPERS@
|
||||||
|
const card = await until(() => all("#main details.repo").find((details) => details.textContent.includes("github.com/me/demo")), 8000);
|
||||||
|
if (!card) {
|
||||||
|
return {error: text(document.getElementById("main"))};
|
||||||
|
}
|
||||||
|
card.open = true;
|
||||||
|
await sleep(300);
|
||||||
|
const row = all(".branch-row", card).find((entry) => text(entry.querySelector(".branch-name strong")) === "feature/squashed");
|
||||||
|
const kept = row ? row.querySelector(".hint.kept") : null;
|
||||||
|
const deletes = all('button[aria-label="Delete"]', card).map((button) => text(button.closest(".branch-row").querySelector(".branch-name strong")));
|
||||||
|
const header = all("#main .delete-all span.state").map(name);
|
||||||
|
document.querySelector('#main button[aria-label="Delete all merged"]').click();
|
||||||
|
const sheet = await until(() => document.querySelector("#sheet:not([hidden]) li.dry-run") && document.getElementById("sheet"), 8000);
|
||||||
|
const kind = (li) => (li.dataset.token ? "token" : li.dataset.held ? "held" : li.dataset.kept ? "kept" : null);
|
||||||
|
const rows = sheet ? all("li.dry-run", sheet).map((li) => [text(li.querySelector("code")), kind(li), all("button", li).length, text(li.querySelector("pre"))]) : null;
|
||||||
|
const confirms = sheet ? all("#sheet button").filter((button) => name(button) === "Delete").length : null;
|
||||||
|
const title = text(document.querySelector(".sheet-head .panel-title"));
|
||||||
|
document.querySelector(".sheet-head button").click();
|
||||||
|
return {kept: kept ? [kept.title, name(kept.querySelector("span.state"))] : null, rowDeletes: row ? all('button[aria-label="Delete"]', row).length : null, deletes, header, rows, confirms, title};
|
||||||
|
})()""".replace("@HELPERS@", HELPERS)
|
||||||
|
KEPT_POPUP = """(async () => {@HELPERS@
|
||||||
|
await screenShown(8000);
|
||||||
|
const picker = await until(() => document.querySelector('#main select[aria-label="Repository"]'), 8000);
|
||||||
|
if (picker && picker.value !== @REPO@) {
|
||||||
|
picker.value = @REPO@;
|
||||||
|
picker.dispatchEvent(new Event("change", {bubbles: true}));
|
||||||
|
}
|
||||||
|
const id = `branch:${@REPO@}\\u0000feature/squashed`;
|
||||||
|
const target = await until(() => all("#main svg [data-pick]").find((node) => node.dataset.pick === id), 8000);
|
||||||
|
if (!target) {
|
||||||
|
return {error: text(document.getElementById("main"))};
|
||||||
|
}
|
||||||
|
target.dispatchEvent(new MouseEvent("click", {bubbles: true, cancelable: true}));
|
||||||
|
const dialog = document.getElementById("popup");
|
||||||
|
const dry = await until(() => dialog.open && dialog.querySelector(".popup-box pre.notes"), 6000);
|
||||||
|
const kept = dialog.querySelector(".popup-box .actions .hint.kept");
|
||||||
|
return {title: text(dialog.querySelector(".popup-box .panel-title")), heading: dry ? text(dry.previousElementSibling) : null, dry: dry ? dry.textContent : null,
|
||||||
|
deletes: all('#popup button[aria-label="Delete"]').length, kept: kept ? kept.title : null};
|
||||||
|
})()""".replace("@HELPERS@", HELPERS)
|
||||||
OVERLAP = """
|
OVERLAP = """
|
||||||
const shown = (node) => node.getClientRects().length > 0 && node.getBoundingClientRect().width > 0;
|
const shown = (node) => node.getClientRects().length > 0 && node.getBoundingClientRect().width > 0;
|
||||||
const words = (label) => [...label.childNodes].filter((node) => node.nodeType === 3).map((node) => node.textContent).join("").trim();
|
const words = (label) => [...label.childNodes].filter((node) => node.nodeType === 3).map((node) => node.textContent).join("").trim();
|
||||||
@@ -1884,7 +1923,7 @@ class TestOrchestratorE2E(HubCase):
|
|||||||
self.assertIn(["twig", "false"], by_hash["#agents/tree"]["spawn"])
|
self.assertIn(["twig", "false"], by_hash["#agents/tree"]["spawn"])
|
||||||
artifacts = by_hash["#artifacts/tree"]
|
artifacts = by_hash["#artifacts/tree"]
|
||||||
self.assertEqual(artifacts["deletes"], [["Delete all merged", True, "Delete all merged blocked: Secret Service locked"]])
|
self.assertEqual(artifacts["deletes"], [["Delete all merged", True, "Delete all merged blocked: Secret Service locked"]])
|
||||||
for state in ("merged by #12 into main", "upstream gone", "protected", "1 commits unpushed", "clean", "not committed: 0 staged, 0 changed, 1 new", "gc skips: ignored files beyond known caches: notes.txt"):
|
for state in ("merged by pr#12 into main", "upstream gone", "protected", "1 commits unpushed", "clean", "not committed: 0 staged, 0 changed, 1 new", "gc skips: ignored files beyond known caches: notes.txt"):
|
||||||
with self.subTest(state=state):
|
with self.subTest(state=state):
|
||||||
self.assertIn(state, artifacts["states"])
|
self.assertIn(state, artifacts["states"])
|
||||||
for view in ("orbit", "chord"):
|
for view in ("orbit", "chord"):
|
||||||
@@ -2237,15 +2276,16 @@ class TestOrchestratorE2E(HubCase):
|
|||||||
def test_chromium_deletes_a_merged_branch_after_dry_run_and_signature(self):
|
def test_chromium_deletes_a_merged_branch_after_dry_run_and_signature(self):
|
||||||
tree = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
tree = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
||||||
tree["delete_ready"], tree["delete_reason"] = True, ""
|
tree["delete_ready"], tree["delete_reason"] = True, ""
|
||||||
next(item for item in tree["repos"][0]["items"] if item["branch"] == "feature/done").update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[])
|
next(item for item in tree["repos"][0]["items"] if item["branch"] == "feature/done").update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[], merged_by="ancestor")
|
||||||
self.tree_file.write_text(json.dumps(tree), encoding="utf-8")
|
self.tree_file.write_text(json.dumps(tree), encoding="utf-8")
|
||||||
|
self.write_local_plan(tree, {})
|
||||||
demo = f"{self.root}/repos/github.com/me/demo"
|
demo = f"{self.root}/repos/github.com/me/demo"
|
||||||
Path(demo).mkdir(parents=True)
|
Path(demo).mkdir(parents=True)
|
||||||
fake = self.outside / "bin" / "pkgmgr"
|
fake = self.outside / "bin" / "pkgmgr"
|
||||||
fake.parent.mkdir()
|
fake.parent.mkdir()
|
||||||
fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8")
|
fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8")
|
||||||
fake.chmod(0o755)
|
fake.chmod(0o755)
|
||||||
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "feature/done", "worktree": None, "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "skip": None, "actions": ["local-branch"]}]), encoding="utf-8")
|
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "feature/done", "worktree": None, "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"]}]), encoding="utf-8")
|
||||||
following = json.loads(json.dumps(tree))
|
following = json.loads(json.dumps(tree))
|
||||||
following["repos"][0]["items"] = [item for item in following["repos"][0]["items"] if item["branch"] != "feature/done"]
|
following["repos"][0]["items"] = [item for item in following["repos"][0]["items"] if item["branch"] != "feature/done"]
|
||||||
(fake.parent / "next-tree.json").write_text(json.dumps(following), encoding="utf-8")
|
(fake.parent / "next-tree.json").write_text(json.dumps(following), encoding="utf-8")
|
||||||
@@ -2269,7 +2309,7 @@ class TestOrchestratorE2E(HubCase):
|
|||||||
self.assertEqual(opened["buttons"], [[False, "Delete: remove feature/done locally, after dry run and signature"]])
|
self.assertEqual(opened["buttons"], [[False, "Delete: remove feature/done locally, after dry run and signature"]])
|
||||||
self.assertEqual(opened["title"], "Delete feature/done")
|
self.assertEqual(opened["title"], "Delete feature/done")
|
||||||
self.assertEqual(len(opened["dry"]), 1)
|
self.assertEqual(len(opened["dry"]), 1)
|
||||||
self.assertTrue(opened["dry"][0].startswith("feature/done @ 5b1e6fd3a7c0: merged-clean, merged by #12"), opened["dry"])
|
self.assertTrue(opened["dry"][0].startswith("feature/done @ 5b1e6fd3a7c0: merged-clean, merged by ancestor"), opened["dry"])
|
||||||
self.assertIn("stays: remote-branch (local only; the host refuses a plan that deletes it)", opened["dry"][0])
|
self.assertIn("stays: remote-branch (local only; the host refuses a plan that deletes it)", opened["dry"][0])
|
||||||
self.assertEqual((confirmed["result"], confirmed["text"]), ("done", "feature/donedeleted"), confirmed)
|
self.assertEqual((confirmed["result"], confirmed["text"]), ("done", "feature/donedeleted"), confirmed)
|
||||||
self.assertEqual(confirmed["toasts"], ["Delete: donefeature/done: deleted"])
|
self.assertEqual(confirmed["toasts"], ["Delete: donefeature/done: deleted"])
|
||||||
@@ -2279,11 +2319,49 @@ class TestOrchestratorE2E(HubCase):
|
|||||||
self.assertEqual([[(item["branch"], item["status"]) for item in result["items"]] for result in written], [[("feature/done", "done")]])
|
self.assertEqual([[(item["branch"], item["status"]) for item in result["items"]] for result in written], [[("feature/done", "done")]])
|
||||||
calls = [json.loads(line)["argv"] for line in (fake.parent / "calls.jsonl").read_text(encoding="utf-8").splitlines()]
|
calls = [json.loads(line)["argv"] for line in (fake.parent / "calls.jsonl").read_text(encoding="utf-8").splitlines()]
|
||||||
self.assertEqual(calls, [
|
self.assertEqual(calls, [
|
||||||
["branch", "cleanup", "--json", "--branch", "feature/done"],
|
["branch", "cleanup", "--local", "--json", "--branch", "feature/done"],
|
||||||
["branch", "cleanup", "--apply", "--json", "--branch", "feature/done"],
|
["branch", "cleanup", "--apply", "--local", "--json", "--branch", "feature/done"],
|
||||||
["branch", "tree", "--format", "json", "--output", str(self.tree_file)],
|
["branch", "tree", "--format", "json", "--output", str(self.tree_file)],
|
||||||
])
|
])
|
||||||
|
|
||||||
|
def test_chromium_shows_a_branch_the_local_dry_run_keeps_without_delete_and_offers_one_whose_local_tip_it_deletes(self):
|
||||||
|
squashed_tip, cherry_tip, cherry_remote = "9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b", "8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b9c", "7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b9c8d"
|
||||||
|
tree = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
||||||
|
tree["delete_ready"], tree["delete_reason"] = True, ""
|
||||||
|
items = tree["repos"][0]["items"]
|
||||||
|
kept = next(item for item in items if item["branch"] == "feature/kept")
|
||||||
|
kept["skip"] = None
|
||||||
|
items.append({**kept, "branch": "feature/squashed", "local_sha": squashed_tip, "upstream": "origin/feature/squashed", "ahead": 0, "behind": 0, "remotes": {"origin": squashed_tip},
|
||||||
|
"merged_by": "pr#7", "actions": ["local-branch", "remote-branch:origin"], "reason": "merged by pr#7"})
|
||||||
|
items.append({**kept, "branch": "feature/cherry", "local_sha": cherry_tip, "upstream": "origin/feature/cherry", "ahead": 0, "behind": 1, "remotes": {"origin": cherry_remote},
|
||||||
|
"merged_by": "patch", "actions": ["local-branch", "remote-branch:origin"], "reason": "merged by patch"})
|
||||||
|
self.tree_file.write_text(json.dumps(tree), encoding="utf-8")
|
||||||
|
self.write_local_plan(tree, {"feature/cherry": {"skip": None, "actions": ["local-branch"]}})
|
||||||
|
demo = f"{self.root}/repos/github.com/me/demo"
|
||||||
|
local = f"http://localhost:{self.port}/"
|
||||||
|
steps = [
|
||||||
|
{"width": 1366, "height": 768, "url": f"{local}#artifacts/tree", "wait": 1200, "probe": KEPT_TREE, "shot": self.shot("1366-kept-tree")},
|
||||||
|
{"width": 1366, "height": 768, "url": f"{local}?kept#artifacts/orbit", "wait": 1200, "probe": KEPT_POPUP.replace("@REPO@", json.dumps(demo)), "shot": self.shot("1366-kept-popup")},
|
||||||
|
]
|
||||||
|
data = self.drive(steps, None)
|
||||||
|
(OUT / "chromium-kept.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
|
||||||
|
self.assertEqual([error for error in data["errors"] if "/favicon.ico" not in error], [])
|
||||||
|
shown, popup = data["results"]
|
||||||
|
self.assertNotIn("error", shown, shown)
|
||||||
|
self.assertNotIn("error", popup, popup)
|
||||||
|
kept_line = "pkgmgr branch cleanup --local keeps it: merged by pr#7, tip is no ancestor of refs/remotes/origin/main; use branch gc (kvb-git-gc, pkgmgr gc)"
|
||||||
|
plan = f"feature/squashed @ {squashed_tip[:12]}: merged-clean, merged by pr#7\n{kept_line}\nbranch gc would delete the local branch feature/squashed\nbranch gc would delete remote-branch:origin"
|
||||||
|
self.assertEqual((shown["kept"], shown["rowDeletes"], shown["deletes"]), ([kept_line, kept_line], 0, ["feature/kept", "feature/cherry"]))
|
||||||
|
self.assertIn("1 kept by the --local dry run: see each dry run", shown["header"])
|
||||||
|
self.assertEqual([row[:3] for row in shown["rows"]], [["feature/kept", None, 0], ["feature/cherry", None, 0], ["feature/done", "held", 0], ["feature/squashed", "kept", 0]])
|
||||||
|
self.assertEqual(shown["rows"][1][3], f"feature/cherry @ {cherry_tip[:12]}: merged-clean, merged by patch\ndeletes the local branch feature/cherry\nstays: remote-branch:origin (local only; the host refuses a plan that deletes it)")
|
||||||
|
self.assertEqual(shown["rows"][3][3], plan)
|
||||||
|
self.assertEqual((shown["confirms"], shown["title"]), (1, "Delete 2 merged branches"))
|
||||||
|
self.assertEqual((popup["title"], popup["heading"], popup["dry"], popup["deletes"], popup["kept"]), ("feature/squashed", "Dry run", plan, 0, kept_line))
|
||||||
|
status, _, body = self.post_json("/api/sign", {"items": [{"kind": "cleanup", "repository": demo, "branch": "feature/squashed", "worktree": "", "tip": squashed_tip}]}, self.guard(self.host))
|
||||||
|
self.assertEqual((status, body.decode()), (400, f"feature/squashed in {demo} is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"))
|
||||||
|
self.assertEqual(list((self.cleanup / "jobs").glob("*.json")), [])
|
||||||
|
|
||||||
def test_chromium_folds_the_hierarchy_by_keyboard_reorders_agents_and_hands_an_office_over_with_a_signature(self):
|
def test_chromium_folds_the_hierarchy_by_keyboard_reorders_agents_and_hands_an_office_over_with_a_signature(self):
|
||||||
signer = self.pin()
|
signer = self.pin()
|
||||||
trust = verifier.Trust(approvers=self.approvers, replay=self.replay / "offices.json", origin=self.origin)
|
trust = verifier.Trust(approvers=self.approvers, replay=self.replay / "offices.json", origin=self.origin)
|
||||||
|
|||||||
@@ -242,8 +242,40 @@ class HubCase(unittest.TestCase):
|
|||||||
root = self.outside / "scan"
|
root = self.outside / "scan"
|
||||||
sample = (Path(__file__).parent / fixture).read_text(encoding="utf-8")
|
sample = (Path(__file__).parent / fixture).read_text(encoding="utf-8")
|
||||||
self.tree_file.write_text(sample.replace("@ROOT@", str(root)), encoding="utf-8")
|
self.tree_file.write_text(sample.replace("@ROOT@", str(root)), encoding="utf-8")
|
||||||
|
self.write_local_plan(json.loads(self.tree_file.read_text(encoding="utf-8")), {})
|
||||||
return root
|
return root
|
||||||
|
|
||||||
|
def write_local_plan(self, tree: dict, local: dict[str, dict]) -> None:
|
||||||
|
"""Args:
|
||||||
|
tree: a tree file's data; each branch becomes the item `pkgmgr
|
||||||
|
branch cleanup --local --json` plans for it as gc.py
|
||||||
|
_local_actions does: a merged-clean local branch without
|
||||||
|
worktree merged by ancestry gets exactly local-branch, any
|
||||||
|
other merged-clean one the skip pkgmgr gives a tip that is no
|
||||||
|
ancestor of the default branch.
|
||||||
|
local: branch -> fields that replace the modelled ones, for a
|
||||||
|
local tip that is an ancestor while the tree's merged_by,
|
||||||
|
which covers every tip, is not.
|
||||||
|
"""
|
||||||
|
repos = []
|
||||||
|
for entry in tree["repos"]:
|
||||||
|
default = f"refs/remotes/origin/{entry.get('default')}"
|
||||||
|
items = []
|
||||||
|
for item in entry["items"]:
|
||||||
|
if not item["branch"]:
|
||||||
|
continue
|
||||||
|
candidate = item["state"] == "merged-clean" and bool(item["local_sha"]) and not item["worktree"]
|
||||||
|
ancestry = candidate and item["merged_by"] == "ancestor"
|
||||||
|
planned = {
|
||||||
|
"branch": item["branch"], "worktree": item["worktree"], "head": item["head"], "local_sha": item["local_sha"], "remotes": item["remotes"], "upstream": item["upstream"], "state": item["state"],
|
||||||
|
"merged_by": item["merged_by"], "merged_into": default if item["state"] == "merged-clean" else None,
|
||||||
|
"skip": f"merged by {item['merged_by']}, tip is no ancestor of {default}" if candidate and not ancestry else None,
|
||||||
|
"actions": ["local-branch"] if ancestry else [], "result": None,
|
||||||
|
}
|
||||||
|
items.append({**planned, **local.get(item["branch"], {})})
|
||||||
|
repos.append({"repo": entry["path"], "default": default, "merged_into": [], "warnings": [], "items": items})
|
||||||
|
self.tree_file.with_name("local-plan.json").write_text(json.dumps({"repos": repos}), encoding="utf-8")
|
||||||
|
|
||||||
def jsonl(self, rows: list[dict]) -> str:
|
def jsonl(self, rows: list[dict]) -> str:
|
||||||
return "".join(json.dumps(row) + "\n" for row in rows)
|
return "".join(json.dumps(row) + "\n" for row in rows)
|
||||||
|
|
||||||
|
|||||||
@@ -106,7 +106,7 @@ class TestHubArtifacts(HubCase):
|
|||||||
self.assertEqual(branches["feature"]["worktrees"][0]["artefacts"], [])
|
self.assertEqual(branches["feature"]["worktrees"][0]["artefacts"], [])
|
||||||
self.assertEqual([(task["task"], [leaf["ref"] for leaf in task["artefacts"]]) for task in data["tasks"]], [("work/loose", ["log:/tmp/loose.log"])])
|
self.assertEqual([(task["task"], [leaf["ref"] for leaf in task["artefacts"]]) for task in data["tasks"]], [("work/loose", ["log:/tmp/loose.log"])])
|
||||||
self.assertEqual(repo["overlaps"], [{"a": "feature", "b": "feature/done", "count": 1, "files": ["a.py"]}])
|
self.assertEqual(repo["overlaps"], [{"a": "feature", "b": "feature/done", "count": 1, "files": ["a.py"]}])
|
||||||
self.assertEqual(repo["merges"], [{"from": "feature/done", "into": "main", "by": "#12"}, {"from": "feature/kept", "into": "main", "by": "ancestor"}])
|
self.assertEqual(repo["merges"], [{"from": "feature/done", "into": "main", "by": "pr#12"}, {"from": "feature/kept", "into": "main", "by": "ancestor"}])
|
||||||
self.assertEqual((branches["feature"]["files"], branches["feature/done"]["files"]), (2, 2))
|
self.assertEqual((branches["feature"]["files"], branches["feature/done"]["files"]), (2, 2))
|
||||||
kinds = [(event["kind"], event["ok"]) for event in branches["feature/done"]["events"]]
|
kinds = [(event["kind"], event["ok"]) for event in branches["feature/done"]["events"]]
|
||||||
self.assertEqual(kinds, [("gate", True), ("review", None), ("gate", True), ("commit", True)])
|
self.assertEqual(kinds, [("gate", True), ("review", None), ("gate", True), ("commit", True)])
|
||||||
|
|||||||
+419
-28
@@ -16,6 +16,7 @@ from pathlib import Path
|
|||||||
|
|
||||||
from hub_case import HubCase
|
from hub_case import HubCase
|
||||||
from hub_support import GIT_ENV, HUB, ORIGIN, key, signing, verifier
|
from hub_support import GIT_ENV, HUB, ORIGIN, key, signing, verifier
|
||||||
|
from kvb_orchestrator import tree
|
||||||
|
|
||||||
CLEANUP = HUB / "client" / "cleanup.py"
|
CLEANUP = HUB / "client" / "cleanup.py"
|
||||||
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_runner", CLEANUP)
|
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_runner", CLEANUP)
|
||||||
@@ -24,24 +25,28 @@ CLEANUP_SPEC.loader.exec_module(host)
|
|||||||
INSTALLED_PKGMGR = shutil.which("pkgmgr")
|
INSTALLED_PKGMGR = shutil.which("pkgmgr")
|
||||||
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
|
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
|
||||||
OTHER_TIP = "1111111111111111111111111111111111111111"
|
OTHER_TIP = "1111111111111111111111111111111111111111"
|
||||||
# Param: format with python=<interpreter>; parses argv as pkgmgr's branch_cmd.py does (argparse exits 2 on any other flag) and prints the plan as gc.py _render does, {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}; plan.json beside it is the plan's items, next-tree.json the rewritten tree, outcome.txt the result --apply reports per item (done without it), exit 0 also for a skipped item.
|
# Param: format with python=<interpreter>; parses argv as pkgmgr's branch_cmd.py does (argparse exits 2 on any other flag) and prints the plan as gc.py _render does, {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}; plan.json beside it is the plan's items, apply.json the items --apply plans instead when present, next-tree.json the rewritten tree, outcome.txt the result --apply reports per item without skip (done without it; an item with skip gets none, as pkgmgr applies only eligible items), killed makes --apply exit 137 without output, unknown.json {"plan": [names], "apply": [names]} the branches that vanished before that run (pkgmgr's "Unknown --branch: a, b" on stderr and exit 2 when one is asked for, else left out of the plan), slow.txt names a cwd it sleeps 30 s in before planning, exit 0 also for a skipped item.
|
||||||
FAKE_PKGMGR = """#!{python}
|
FAKE_PKGMGR = """#!{python}
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
here = Path(__file__).parent
|
here = Path(__file__).parent
|
||||||
with (here / "calls.jsonl").open("a", encoding="utf-8") as log:
|
with (here / "calls.jsonl").open("a", encoding="utf-8") as log:
|
||||||
log.write(json.dumps({{"argv": sys.argv[1:], "cwd": os.getcwd()}}) + "\\n")
|
log.write(json.dumps({{"argv": sys.argv[1:], "cwd": os.getcwd()}}) + "\\n")
|
||||||
|
slow = here / "slow.txt"
|
||||||
|
if slow.exists() and slow.read_text(encoding="utf-8") == os.getcwd():
|
||||||
|
time.sleep(30)
|
||||||
parser = argparse.ArgumentParser(prog="pkgmgr")
|
parser = argparse.ArgumentParser(prog="pkgmgr")
|
||||||
branch = parser.add_subparsers(dest="command", required=True).add_parser("branch").add_subparsers(dest="subcommand", required=True)
|
branch = parser.add_subparsers(dest="command", required=True).add_parser("branch").add_subparsers(dest="subcommand", required=True)
|
||||||
tree = branch.add_parser("tree")
|
tree = branch.add_parser("tree")
|
||||||
tree.add_argument("--format", choices=("json", "yaml"), default="json")
|
tree.add_argument("--format", choices=("json", "yaml"), default="json")
|
||||||
tree.add_argument("--output")
|
tree.add_argument("--output")
|
||||||
cleanup = branch.add_parser("cleanup")
|
cleanup = branch.add_parser("cleanup")
|
||||||
for flag in ("--apply", "--json", "--all", "--sizes"):
|
for flag in ("--apply", "--json", "--local", "--all", "--sizes"):
|
||||||
cleanup.add_argument(flag, action="store_true")
|
cleanup.add_argument(flag, action="store_true")
|
||||||
for option in ("--branch", "--exclude", "--merged-into"):
|
for option in ("--branch", "--exclude", "--merged-into"):
|
||||||
cleanup.add_argument(option, action="append", default=[])
|
cleanup.add_argument(option, action="append", default=[])
|
||||||
@@ -53,31 +58,112 @@ if args.subcommand == "tree":
|
|||||||
Path(args.output).write_text(rewritten, encoding="utf-8")
|
Path(args.output).write_text(rewritten, encoding="utf-8")
|
||||||
raise SystemExit(0)
|
raise SystemExit(0)
|
||||||
items = json.loads((here / "plan.json").read_text(encoding="utf-8"))
|
items = json.loads((here / "plan.json").read_text(encoding="utf-8"))
|
||||||
|
if args.apply and (here / "apply.json").exists():
|
||||||
|
items = json.loads((here / "apply.json").read_text(encoding="utf-8"))
|
||||||
|
unknown = here / "unknown.json"
|
||||||
|
vanished = json.loads(unknown.read_text(encoding="utf-8")).get("apply" if args.apply else "plan", []) if unknown.exists() else []
|
||||||
|
missing = [name for name in args.branch if name in vanished]
|
||||||
|
if missing:
|
||||||
|
print(f"Unknown --branch: {{', '.join(missing)}}", file=sys.stderr)
|
||||||
|
raise SystemExit(2)
|
||||||
|
items = [item for item in items if item["branch"] not in vanished]
|
||||||
|
if args.apply and (here / "killed").exists():
|
||||||
|
raise SystemExit(137)
|
||||||
if args.apply:
|
if args.apply:
|
||||||
outcome = here / "outcome.txt"
|
outcome = here / "outcome.txt"
|
||||||
result = outcome.read_text(encoding="utf-8") if outcome.exists() else "done"
|
result = outcome.read_text(encoding="utf-8") if outcome.exists() else "done"
|
||||||
items = [{{**item, "result": result}} for item in items]
|
items = [item if item.get("skip") else {{**item, "result": result}} for item in items]
|
||||||
plan = {{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}}]}}
|
plan = {{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}}]}}
|
||||||
print(json.dumps(plan) if args.json else "\\n".join(str(item["branch"]) for item in items))
|
print(json.dumps(plan) if args.json else "\\n".join(str(item["branch"]) for item in items))
|
||||||
"""
|
"""
|
||||||
|
RECORDED = Path(__file__).parent / "branch-cleanup-local.json"
|
||||||
|
RECORDED_REPO = "@REPO@"
|
||||||
|
CAPTURE_DATE = "2026-11-14T12:00:00+00:00"
|
||||||
|
KEPT_PATCH = "merged by patch, tip is no ancestor of refs/remotes/origin/main"
|
||||||
|
|
||||||
|
|
||||||
|
def capture_local_plan(base: Path, pkgmgr: Path) -> tuple[Path, dict]:
|
||||||
|
"""Args:
|
||||||
|
base: an empty directory for the origin and the repository.
|
||||||
|
pkgmgr: the pkgmgr to plan with.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The repository and the plan pkgmgr prints for `branch cleanup
|
||||||
|
--local --json` in it, built with fixed dates so every sha repeats:
|
||||||
|
side merged by ancestry; squashed squash-merged, so its tip is no
|
||||||
|
ancestor of main; topic, whose local tip is an ancestor of main
|
||||||
|
while its remote tip was cherry-picked onto main.
|
||||||
|
"""
|
||||||
|
env = {**os.environ, **GIT_ENV, "GIT_AUTHOR_DATE": CAPTURE_DATE, "GIT_COMMITTER_DATE": CAPTURE_DATE}
|
||||||
|
|
||||||
|
def git(cwd: Path, *args: str) -> str:
|
||||||
|
return subprocess.run(["git", *args], cwd=cwd, env=env, check=True, capture_output=True, text=True).stdout.strip()
|
||||||
|
|
||||||
|
def commit(repo: Path, name: str) -> str:
|
||||||
|
(repo / f"{name}.txt").write_text(name, encoding="utf-8")
|
||||||
|
git(repo, "add", f"{name}.txt")
|
||||||
|
git(repo, "commit", "-q", "-m", name)
|
||||||
|
return git(repo, "rev-parse", "HEAD")
|
||||||
|
|
||||||
|
origin, repo = base / "origin.git", base / "repo"
|
||||||
|
git(base, "init", "-q", "--bare", "-b", "main", str(origin))
|
||||||
|
git(base, "init", "-q", "-b", "main", str(repo))
|
||||||
|
git(repo, "remote", "add", "origin", origin.as_uri())
|
||||||
|
git(repo, "commit", "-q", "--allow-empty", "-m", "one")
|
||||||
|
for name in ("side", "squashed", "topic"):
|
||||||
|
git(repo, "checkout", "-q", "-b", name, "main")
|
||||||
|
commit(repo, name)
|
||||||
|
git(repo, "checkout", "-q", "topic")
|
||||||
|
remote_tip = commit(repo, "topic-remote")
|
||||||
|
git(repo, "push", "-q", "origin", "topic:topic")
|
||||||
|
git(repo, "reset", "-q", "--hard", "topic~1")
|
||||||
|
git(repo, "checkout", "-q", "main")
|
||||||
|
git(repo, "merge", "-q", "--no-ff", "-m", "merge side and topic", "side", "topic")
|
||||||
|
git(repo, "merge", "-q", "--squash", "squashed")
|
||||||
|
git(repo, "commit", "-q", "-m", "squashed as one commit")
|
||||||
|
git(repo, "cherry-pick", remote_tip)
|
||||||
|
git(repo, "push", "-q", "origin", "main:main")
|
||||||
|
git(repo, "fetch", "-q", "origin")
|
||||||
|
git(repo, "remote", "set-head", "origin", "main")
|
||||||
|
code, stdout, stderr = host.run(pkgmgr, ["branch", "cleanup", "--local", "--json"], str(repo), host.TIMEOUT)
|
||||||
|
if code != 0:
|
||||||
|
raise AssertionError(stderr)
|
||||||
|
return repo, json.loads(stdout)
|
||||||
|
|
||||||
|
|
||||||
|
def recorded_plan(repo: str) -> dict:
|
||||||
|
"""Returns:
|
||||||
|
The plan the installed pkgmgr printed for capture_local_plan(),
|
||||||
|
recorded in branch-cleanup-local.json, its repository named repo.
|
||||||
|
"""
|
||||||
|
return json.loads(RECORDED.read_text(encoding="utf-8").replace(RECORDED_REPO, repo))
|
||||||
|
|
||||||
|
|
||||||
class TestHubCleanup(HubCase):
|
class TestHubCleanup(HubCase):
|
||||||
def ready(self, ready: bool, held: bool) -> dict:
|
def ready(self, ready: bool, held: bool, merged_by: str, local: dict[str, dict]) -> dict:
|
||||||
"""Args:
|
"""Args:
|
||||||
ready: delete_ready of the tree file.
|
ready: delete_ready of the tree file.
|
||||||
held: keep feature/done in its worktree; otherwise it is a local
|
held: keep feature/done in its worktree; otherwise it is a local
|
||||||
branch without one.
|
branch without one.
|
||||||
|
merged_by: what merged feature/done over all its tips, as pkgmgr
|
||||||
|
gc.py _merged_by names it: ancestor, pr#N, line:X or patch.
|
||||||
|
local: branch -> fields of its `--local` dry run item that
|
||||||
|
replace the ones HubCase.write_local_plan models.
|
||||||
"""
|
"""
|
||||||
root = self.write_tree()
|
root = self.write_tree()
|
||||||
data = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
data = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
||||||
data["delete_ready"], data["delete_reason"] = ready, "" if ready else "Secret Service locked"
|
data["delete_ready"], data["delete_reason"] = ready, "" if ready else "Secret Service locked"
|
||||||
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
|
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
|
||||||
|
done["merged_by"] = merged_by
|
||||||
if not held:
|
if not held:
|
||||||
done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[])
|
done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], remotes={"origin": DONE_TIP}, compose_projects=[], images=[])
|
||||||
self.tree_file.write_text(json.dumps(data), encoding="utf-8")
|
self.tree_file.write_text(json.dumps(data), encoding="utf-8")
|
||||||
|
self.write_local_plan(data, local)
|
||||||
return signing.cleanup_item(f"{root}/repos/github.com/me/demo", "feature/done", done["worktree"] or "", DONE_TIP)
|
return signing.cleanup_item(f"{root}/repos/github.com/me/demo", "feature/done", done["worktree"] or "", DONE_TIP)
|
||||||
|
|
||||||
|
def refusal(self, item: dict) -> str:
|
||||||
|
return f"{item['branch']} in {item['repository']} is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"
|
||||||
|
|
||||||
def branches(self) -> dict:
|
def branches(self) -> dict:
|
||||||
status, _, body = self.get("/api/tree")
|
status, _, body = self.get("/api/tree")
|
||||||
self.assertEqual(status, 200, body)
|
self.assertEqual(status, 200, body)
|
||||||
@@ -89,24 +175,109 @@ class TestHubCleanup(HubCase):
|
|||||||
return status, body
|
return status, body
|
||||||
|
|
||||||
def test_the_tree_offers_a_delete_only_for_merged_clean_branches_without_skip_reason(self):
|
def test_the_tree_offers_a_delete_only_for_merged_clean_branches_without_skip_reason(self):
|
||||||
item = self.ready(False, False)
|
item = self.ready(False, False, "ancestor", {})
|
||||||
branches = self.branches()
|
branches = self.branches()
|
||||||
done = branches["feature/done"]
|
done = branches["feature/done"]
|
||||||
self.assertEqual((done["cleanup"], done["held"]), (item, ""))
|
self.assertEqual((done["cleanup"], done["held"], done["kept"]), (item, "", ""))
|
||||||
self.assertEqual(done["plan"], [
|
self.assertEqual(done["plan"], [
|
||||||
"feature/done @ 5b1e6fd3a7c0: merged-clean, merged by #12",
|
"feature/done @ 5b1e6fd3a7c0: merged-clean, merged by ancestor",
|
||||||
"deletes the local branch feature/done",
|
"deletes the local branch feature/done",
|
||||||
"stays: remote-branch (local only; the host refuses a plan that deletes it)",
|
"stays: remote-branch (local only; the host refuses a plan that deletes it)",
|
||||||
])
|
])
|
||||||
self.assertEqual((done["merged_by"], done["merged_into"], done["gone"]), ("#12", "main", True))
|
self.assertEqual((done["merged_by"], done["merged_into"], done["gone"]), ("ancestor", "main", True))
|
||||||
self.assertEqual({name: entry["cleanup"] for name, entry in branches.items() if name != "feature/done"}, {"feature": None, "feature/kept": None, "main": None})
|
self.assertEqual({name: entry["cleanup"] for name, entry in branches.items() if name != "feature/done"}, {"feature": None, "feature/kept": None, "main": None})
|
||||||
self.assertEqual(branches["feature/kept"]["skip"], "ignored files beyond known caches: notes.txt")
|
self.assertEqual(branches["feature/kept"]["skip"], "ignored files beyond known caches: notes.txt")
|
||||||
|
|
||||||
|
def test_the_local_dry_run_decides_by_its_actions_not_by_what_merged_the_branch(self):
|
||||||
|
deletes = {"state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"]}
|
||||||
|
cases = (
|
||||||
|
("ancestry", deletes, ""),
|
||||||
|
("local tip an ancestor, remote tip patch-merged", {**deletes, "merged_by": "patch"}, ""),
|
||||||
|
("tip no ancestor", {**deletes, "merged_by": "patch", "skip": KEPT_PATCH, "actions": []}, KEPT_PATCH),
|
||||||
|
("not merged", {**deletes, "state": "unmerged", "actions": []}, "state is unmerged, not merged-clean"),
|
||||||
|
("no state", {"actions": ["local-branch"]}, "state is None, not merged-clean"),
|
||||||
|
("remote delete", {**deletes, "actions": ["local-branch", "remote-branch:origin"]}, "the plan also deletes remote-branch:origin; a signed delete is local only"),
|
||||||
|
("nothing planned", {**deletes, "actions": []}, "the dry run plans no delete, not exactly the local branch"),
|
||||||
|
("no actions", {key: value for key, value in deletes.items() if key != "actions"}, "the dry run plans no delete, not exactly the local branch"),
|
||||||
|
("worktree too", {**deletes, "actions": ["worktree", "local-branch"]}, "the dry run plans worktree, local-branch, not exactly the local branch"),
|
||||||
|
)
|
||||||
|
for name, entry, kept in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
self.assertEqual(signing.local_keeps(entry), kept)
|
||||||
|
|
||||||
|
def test_a_branch_the_local_dry_run_keeps_offers_no_delete_and_its_dry_run_says_why_from_that_dry_run(self):
|
||||||
|
for merged_by in ("pr#12", "patch"):
|
||||||
|
with self.subTest(merged_by=merged_by):
|
||||||
|
item = self.ready(True, False, merged_by, {})
|
||||||
|
done = self.branches()["feature/done"]
|
||||||
|
kept = f"pkgmgr branch cleanup --local keeps it: merged by {merged_by}, tip is no ancestor of refs/remotes/origin/main; use branch gc (kvb-git-gc, pkgmgr gc)"
|
||||||
|
self.assertEqual((done["cleanup"], done["held"], done["kept"], done["token_free"]), (None, "", kept, False))
|
||||||
|
self.assertEqual(done["plan"], [
|
||||||
|
f"feature/done @ 5b1e6fd3a7c0: merged-clean, merged by {merged_by}",
|
||||||
|
kept,
|
||||||
|
"branch gc would delete the local branch feature/done",
|
||||||
|
"branch gc would delete remote-branch",
|
||||||
|
])
|
||||||
|
status, body = self.sign([item])
|
||||||
|
self.assertEqual((status, body.decode()), (400, self.refusal(item)))
|
||||||
|
page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('if (entry.kept) {\n return h("span", {class: "hint kept", title: entry.kept}, chip("bi-sign-merge-left", entry.kept, "neutral"));\n }\n if (!entry.cleanup) {', page)
|
||||||
|
self.assertIn("!entry.held && !entry.kept && (entry.cleanup", page)
|
||||||
|
self.assertIn("deleteDialog(items, locked, held, kept)", page)
|
||||||
|
self.assertIn('keeps ? chip("bi-sign-merge-left", "kept by the --local dry run: not deleted here", "neutral")', (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
def test_a_branch_whose_local_tip_the_local_dry_run_deletes_is_offered_although_its_remote_tip_was_patch_merged(self):
|
||||||
|
item = self.ready(True, False, "patch", {"feature/done": {"skip": None, "actions": ["local-branch"]}})
|
||||||
|
done = self.branches()["feature/done"]
|
||||||
|
self.assertEqual((done["cleanup"], done["kept"], done["merged_by"]), (item, "", "patch"))
|
||||||
|
self.assertEqual(done["plan"][1], "deletes the local branch feature/done")
|
||||||
|
status, body = self.sign([item])
|
||||||
|
self.assertEqual(status, 200, body)
|
||||||
|
|
||||||
|
def test_a_branch_another_branch_tracks_as_local_upstream_is_kept_as_pkgmgr_keeps_it_when_the_host_chooses_it_alone(self):
|
||||||
|
kept = "pkgmgr branch cleanup --local keeps it: kept: local upstream of feature; use branch gc (kvb-git-gc, pkgmgr gc)"
|
||||||
|
cases = (
|
||||||
|
("local upstream", {"feature": {"upstream": "feature/done"}}, kept),
|
||||||
|
("local upstream git shortens with heads/", {"feature": {"upstream": "heads/feature/done"}}, kept),
|
||||||
|
("remote upstream of the same name", {"feature": {"upstream": "origin/feature/done"}}, ""),
|
||||||
|
("its own upstream", {"feature/done": {"upstream": "feature/done"}}, ""),
|
||||||
|
)
|
||||||
|
for name, local, expected in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
item = self.ready(True, False, "ancestor", local)
|
||||||
|
done = self.branches()["feature/done"]
|
||||||
|
self.assertEqual((done["cleanup"], done["kept"]), (None, expected) if expected else (item, ""))
|
||||||
|
if expected:
|
||||||
|
self.assertEqual(done["plan"][1], expected)
|
||||||
|
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
|
||||||
|
|
||||||
|
def test_a_repository_whose_replan_did_not_finish_offers_no_delete_and_says_why(self):
|
||||||
|
item = self.ready(True, False, "ancestor", {})
|
||||||
|
planned = self.tree_file.with_name("local-plan.json")
|
||||||
|
data = json.loads(planned.read_text(encoding="utf-8"))
|
||||||
|
data["repos"][0]["stale"] = "replan timed out at 2026-11-14T18:00:00+00:00"
|
||||||
|
planned.write_text(json.dumps(data), encoding="utf-8")
|
||||||
|
done = self.branches()["feature/done"]
|
||||||
|
stale = "pkgmgr branch cleanup --local plan stale: replan timed out at 2026-11-14T18:00:00+00:00; use branch gc (kvb-git-gc, pkgmgr gc)"
|
||||||
|
self.assertEqual((done["cleanup"], done["kept"], done["plan"][1]), (None, stale, stale))
|
||||||
|
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
|
||||||
|
|
||||||
|
def test_without_a_local_dry_run_of_the_tip_the_tree_offers_no_delete(self):
|
||||||
|
pending = "no pkgmgr branch cleanup --local dry run of 5b1e6fd3a7c0 yet: the tree timer writes local-plan.json within a minute"
|
||||||
|
for name, local in (("dry run of another tip", {"feature/done": {"local_sha": OTHER_TIP}}), ("no dry run file", None)):
|
||||||
|
with self.subTest(name=name):
|
||||||
|
item = self.ready(True, False, "ancestor", local or {})
|
||||||
|
if local is None:
|
||||||
|
self.tree_file.with_name("local-plan.json").unlink()
|
||||||
|
done = self.branches()["feature/done"]
|
||||||
|
self.assertEqual((done["cleanup"], done["kept"], done["plan"][1]), (None, pending, pending))
|
||||||
|
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
|
||||||
|
|
||||||
def test_signing_a_delete_needs_delete_ready_and_exactly_an_offered_item(self):
|
def test_signing_a_delete_needs_delete_ready_and_exactly_an_offered_item(self):
|
||||||
item = self.ready(False, False)
|
item = self.ready(False, False, "ancestor", {})
|
||||||
status, body = self.sign([item])
|
status, body = self.sign([item])
|
||||||
self.assertEqual((status, body.decode()), (400, "delete blocked: Secret Service locked\n"))
|
self.assertEqual((status, body.decode()), (400, "delete blocked: Secret Service locked\n"))
|
||||||
item = self.ready(True, False)
|
item = self.ready(True, False, "ancestor", {})
|
||||||
kept = {**item, "branch": "feature/kept", "worktree": "", "tip": "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"}
|
kept = {**item, "branch": "feature/kept", "worktree": "", "tip": "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"}
|
||||||
for name, items in (("changed tip", [{**item, "tip": OTHER_TIP}]), ("skip reason", [kept]), ("pattern", [{**item, "branch": "feature/*"}]), ("twice", [item, item]), ("extra key", [{**item, "command": "rm -rf /"}])):
|
for name, items in (("changed tip", [{**item, "tip": OTHER_TIP}]), ("skip reason", [kept]), ("pattern", [{**item, "branch": "feature/*"}]), ("twice", [item, item]), ("extra key", [{**item, "command": "rm -rf /"}])):
|
||||||
with self.subTest(name=name):
|
with self.subTest(name=name):
|
||||||
@@ -116,15 +287,15 @@ class TestHubCleanup(HubCase):
|
|||||||
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(item)])
|
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(item)])
|
||||||
|
|
||||||
def test_a_branch_held_by_a_worktree_offers_no_delete_and_names_branch_gc(self):
|
def test_a_branch_held_by_a_worktree_offers_no_delete_and_names_branch_gc(self):
|
||||||
item = self.ready(True, True)
|
item = self.ready(True, True, "pr#12", {})
|
||||||
done = self.branches()["feature/done"]
|
done = self.branches()["feature/done"]
|
||||||
held = f"held by worktree {item['worktree']}: pkgmgr branch cleanup skips it, use branch gc (kvb-git-gc, pkgmgr gc)"
|
held = f"held by worktree {item['worktree']}: pkgmgr branch cleanup skips it, use branch gc (kvb-git-gc, pkgmgr gc)"
|
||||||
self.assertEqual((done["cleanup"], done["held"], done["plan"][1]), (None, held, held))
|
self.assertEqual((done["cleanup"], done["held"], done["kept"], done["plan"][1]), (None, held, "", held))
|
||||||
status, body = self.sign([item])
|
status, body = self.sign([item])
|
||||||
self.assertEqual((status, body.decode()), (400, f"feature/done in {item['repository']} is no merged-clean branch without skip reason at that tip in the tree file\n"))
|
self.assertEqual((status, body.decode()), (400, self.refusal(item)))
|
||||||
page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8")
|
page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8")
|
||||||
self.assertIn('if (entry.held) {\n return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));', page)
|
self.assertIn('if (entry.held) {\n return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));', page)
|
||||||
self.assertIn("deleteDialog(items, locked, held)", page)
|
self.assertIn("deleteDialog(items, locked, held, kept)", page)
|
||||||
self.assertIn("held by a worktree: not deleted here", (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8"))
|
self.assertIn("held by a worktree: not deleted here", (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8"))
|
||||||
|
|
||||||
def test_the_delete_sheet_renders_the_dry_run_note_as_visible_text(self):
|
def test_the_delete_sheet_renders_the_dry_run_note_as_visible_text(self):
|
||||||
@@ -132,7 +303,8 @@ class TestHubCleanup(HubCase):
|
|||||||
sheet = dialog[dialog.index("openSheet("):]
|
sheet = dialog[dialog.index("openSheet("):]
|
||||||
note = re.search(r'\[\n\s*h\("p", \{class: "muted"\}, "([^"]+)"\),\n', sheet)
|
note = re.search(r'\[\n\s*h\("p", \{class: "muted"\}, "([^"]+)"\),\n', sheet)
|
||||||
self.assertIsNotNone(note, sheet[:400])
|
self.assertIsNotNone(note, sheet[:400])
|
||||||
for part in ("it deletes the local branch", "never touches a worktree, Compose stack or volume", "A branch held by a worktree is not deleted here", "its docker compose down -v deletes the stack's volumes", "the host refuses an entry whose plan deletes one", "refuses every entry whose state has changed since"):
|
for part in ("it deletes the local branch", "never touches a worktree, Compose stack or volume", "A branch held by a worktree is not deleted here", "its docker compose down -v deletes the stack's volumes", "the host refuses an entry whose plan deletes one", "refuses every entry whose state has changed since",
|
||||||
|
"Neither is a branch the pkgmgr branch cleanup --local dry run keeps, such as one merged by pull request, squash or patch whose local tip is no ancestor of the default branch: branch gc removes it"):
|
||||||
self.assertIn(part, note.group(1))
|
self.assertIn(part, note.group(1))
|
||||||
self.assertEqual(dialog.count(note.group(1)), 1)
|
self.assertEqual(dialog.count(note.group(1)), 1)
|
||||||
|
|
||||||
@@ -143,7 +315,7 @@ class TestHubCleanup(HubCase):
|
|||||||
downs = [["docker", *(part.value if isinstance(part, ast.Constant) else "{project}" for part in call.args[0].elts)] for call in calls if any(isinstance(part, ast.Constant) and part.value == "down" for part in call.args[0].elts)]
|
downs = [["docker", *(part.value if isinstance(part, ast.Constant) else "{project}" for part in call.args[0].elts)] for call in calls if any(isinstance(part, ast.Constant) and part.value == "down" for part in call.args[0].elts)]
|
||||||
self.assertEqual(len(downs), 1, downs)
|
self.assertEqual(len(downs), 1, downs)
|
||||||
command = " ".join(downs[0]).format(project="demo-feature-done")
|
command = " ".join(downs[0]).format(project="demo-feature-done")
|
||||||
self.ready(True, True)
|
self.ready(True, True, "pr#12", {})
|
||||||
plan = self.branches()["feature/done"]["plan"]
|
plan = self.branches()["feature/done"]["plan"]
|
||||||
self.assertIn(f"branch gc would run {command}, which deletes the volumes of demo-feature-done", plan)
|
self.assertIn(f"branch gc would run {command}, which deletes the volumes of demo-feature-done", plan)
|
||||||
self.assertIn("-v", downs[0])
|
self.assertIn("-v", downs[0])
|
||||||
@@ -154,7 +326,7 @@ class TestHubCleanup(HubCase):
|
|||||||
self.assertFalse([text for text in texts if "without -v" in text])
|
self.assertFalse([text for text in texts if "without -v" in text])
|
||||||
|
|
||||||
def test_a_signed_delete_becomes_a_job_without_any_command(self):
|
def test_a_signed_delete_becomes_a_job_without_any_command(self):
|
||||||
item = self.ready(True, False)
|
item = self.ready(True, False, "ancestor", {})
|
||||||
signer = self.pin()
|
signer = self.pin()
|
||||||
proof = signer.proof(self.prepare([item])["payload"], self.origin)
|
proof = signer.proof(self.prepare([item])["payload"], self.origin)
|
||||||
self.assertEqual(self.post_json("/api/cleanup", {"items": [item], "proof": None}, self.guard(self.host))[0], 400)
|
self.assertEqual(self.post_json("/api/cleanup", {"items": [item], "proof": None}, self.guard(self.host))[0], 400)
|
||||||
@@ -206,7 +378,7 @@ class TestHostCleanup(unittest.TestCase):
|
|||||||
return signer.proof(signing.approval_payload(hashes, secrets.token_hex(16), int(time.time()) + 600), ORIGIN)
|
return signer.proof(signing.approval_payload(hashes, secrets.token_hex(16), int(time.time()) + 600), ORIGIN)
|
||||||
|
|
||||||
def entry(self, **changes: object) -> dict:
|
def entry(self, **changes: object) -> dict:
|
||||||
return {"branch": self.item["branch"], "worktree": self.item["worktree"], "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "skip": None, **changes}
|
return {"branch": self.item["branch"], "worktree": self.item["worktree"], "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"], **changes}
|
||||||
|
|
||||||
def run_job(self, items: list[dict], proof: object, plan: list[dict]) -> tuple[dict, list[dict]]:
|
def run_job(self, items: list[dict], proof: object, plan: list[dict]) -> tuple[dict, list[dict]]:
|
||||||
(self.fake.parent / "plan.json").write_text(json.dumps(plan), encoding="utf-8")
|
(self.fake.parent / "plan.json").write_text(json.dumps(plan), encoding="utf-8")
|
||||||
@@ -221,7 +393,7 @@ class TestHostCleanup(unittest.TestCase):
|
|||||||
def test_a_signed_merged_clean_item_is_dry_run_then_deleted_by_exact_name(self):
|
def test_a_signed_merged_clean_item_is_dry_run_then_deleted_by_exact_name(self):
|
||||||
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
||||||
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")])
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")])
|
||||||
selector = ["--json", "--branch", "feature/done"]
|
selector = ["--local", "--json", "--branch", "feature/done"]
|
||||||
self.assertEqual(calls, [
|
self.assertEqual(calls, [
|
||||||
{"argv": ["branch", "cleanup", *selector], "cwd": str(self.repo)},
|
{"argv": ["branch", "cleanup", *selector], "cwd": str(self.repo)},
|
||||||
{"argv": ["branch", "cleanup", "--apply", *selector], "cwd": str(self.repo)},
|
{"argv": ["branch", "cleanup", "--apply", *selector], "cwd": str(self.repo)},
|
||||||
@@ -239,7 +411,7 @@ class TestHostCleanup(unittest.TestCase):
|
|||||||
outcome.write_text(result, encoding="utf-8")
|
outcome.write_text(result, encoding="utf-8")
|
||||||
done, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
done, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
||||||
self.assertEqual([(entry["status"], entry["reason"], entry["code"]) for entry in done["items"]], [("failed", reason, 0)])
|
self.assertEqual([(entry["status"], entry["reason"], entry["code"]) for entry in done["items"]], [("failed", reason, 0)])
|
||||||
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--json"], ["branch", "cleanup", "--apply"]])
|
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"]])
|
||||||
self.assertNotIn("tree", done)
|
self.assertNotIn("tree", done)
|
||||||
self.assertEqual(self.tree.read_text(encoding="utf-8"), before)
|
self.assertEqual(self.tree.read_text(encoding="utf-8"), before)
|
||||||
outcome.write_text("done; image kept: demo:feature-done", encoding="utf-8")
|
outcome.write_text("done; image kept: demo:feature-done", encoding="utf-8")
|
||||||
@@ -261,7 +433,7 @@ class TestHostCleanup(unittest.TestCase):
|
|||||||
("option name", [option], self.proof([option], self.signer), [self.entry(branch="--all")], "pattern or option characters", 0),
|
("option name", [option], self.proof([option], self.signer), [self.entry(branch="--all")], "pattern or option characters", 0),
|
||||||
("changed tip", [self.item], self.proof([self.item], self.signer), [self.entry(local_sha=OTHER_TIP)], "tip changed", 1),
|
("changed tip", [self.item], self.proof([self.item], self.signer), [self.entry(local_sha=OTHER_TIP)], "tip changed", 1),
|
||||||
("not merged-clean", [self.item], self.proof([self.item], self.signer), [self.entry(state="dirty")], "state is dirty", 1),
|
("not merged-clean", [self.item], self.proof([self.item], self.signer), [self.entry(state="dirty")], "state is dirty", 1),
|
||||||
("skip reason", [self.item], self.proof([self.item], self.signer), [self.entry(skip="ignored files")], "gc skip reason", 1),
|
("skip reason", [self.item], self.proof([self.item], self.signer), [self.entry(skip="ignored files")], "pkgmgr branch cleanup --local keeps it: ignored files", 1),
|
||||||
("other worktree", [self.item], self.proof([self.item], self.signer), [self.entry(worktree="/elsewhere")], "worktree changed", 1),
|
("other worktree", [self.item], self.proof([self.item], self.signer), [self.entry(worktree="/elsewhere")], "worktree changed", 1),
|
||||||
("pattern match in pkgmgr", [self.item], self.proof([self.item], self.signer), [self.entry(), self.entry(branch="feature/done-too")], "other branches too", 1),
|
("pattern match in pkgmgr", [self.item], self.proof([self.item], self.signer), [self.entry(), self.entry(branch="feature/done-too")], "other branches too", 1),
|
||||||
("remote branch delete", [self.item], self.proof([self.item], self.signer), [self.entry(actions=["local-branch", "remote-branch:origin"])], "a signed delete is local only", 1),
|
("remote branch delete", [self.item], self.proof([self.item], self.signer), [self.entry(actions=["local-branch", "remote-branch:origin"])], "a signed delete is local only", 1),
|
||||||
@@ -271,9 +443,125 @@ class TestHostCleanup(unittest.TestCase):
|
|||||||
result, calls = self.run_job(items, proof, plan)
|
result, calls = self.run_job(items, proof, plan)
|
||||||
self.assertEqual([entry["status"] for entry in result["items"]], ["refused"])
|
self.assertEqual([entry["status"] for entry in result["items"]], ["refused"])
|
||||||
self.assertIn(reason, result["items"][0]["reason"])
|
self.assertIn(reason, result["items"][0]["reason"])
|
||||||
self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--json", "--branch", items[0]["branch"]]] * dry_runs)
|
self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--local", "--json", "--branch", items[0]["branch"]]] * dry_runs)
|
||||||
self.assertNotIn("tree", result)
|
self.assertNotIn("tree", result)
|
||||||
|
|
||||||
|
def test_a_signed_item_the_local_dry_run_keeps_is_refused_with_its_reason_and_never_applied(self):
|
||||||
|
cases = (("tip no ancestor", {"merged_by": "patch", "skip": KEPT_PATCH, "actions": []}, KEPT_PATCH), ("nothing planned", {"actions": []}, "the dry run plans no delete, not exactly the local branch"))
|
||||||
|
for name, changes, kept in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry(**changes)])
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("refused", f"pkgmgr branch cleanup --local keeps it: {kept}")])
|
||||||
|
self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--local", "--json", "--branch", "feature/done"]])
|
||||||
|
self.assertNotIn("tree", result)
|
||||||
|
|
||||||
|
def test_a_signed_item_whose_local_tip_the_dry_run_deletes_is_deleted_whatever_merged_its_remote_tip(self):
|
||||||
|
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry(merged_by="patch", remotes={"origin": OTHER_TIP})])
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")])
|
||||||
|
self.assertEqual([call["argv"][:4] for call in calls][:2], [["branch", "cleanup", "--local", "--json"], ["branch", "cleanup", "--apply", "--local"]])
|
||||||
|
|
||||||
|
def test_an_apply_that_deletes_the_branch_at_another_merged_tip_is_done_with_a_warning_and_rewrites_the_tree(self):
|
||||||
|
(self.fake.parent / "apply.json").write_text(json.dumps([self.entry(local_sha=OTHER_TIP, head=OTHER_TIP)]), encoding="utf-8")
|
||||||
|
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "deleted at another merged tip than the signed dry run: tip changed: signed 5b1e6fd3a7c0, now 111111111111")])
|
||||||
|
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"], ["branch", "tree", "--format"]])
|
||||||
|
self.assertEqual((result["tree"], json.loads(self.tree.read_text(encoding="utf-8"))), ({"code": 0, "error": ""}, {"rewritten": True}))
|
||||||
|
|
||||||
|
def test_an_apply_that_skips_the_branch_at_another_tip_is_failed_and_keeps_the_tree(self):
|
||||||
|
before = self.tree.read_text(encoding="utf-8")
|
||||||
|
(self.fake.parent / "apply.json").write_text(json.dumps([self.entry(local_sha=OTHER_TIP, head=OTHER_TIP)]), encoding="utf-8")
|
||||||
|
(self.fake.parent / "outcome.txt").write_text("skipped: changed since plan (merged by patch, tip is no ancestor of refs/remotes/origin/main)", encoding="utf-8")
|
||||||
|
result, _ = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("failed", "the apply planned another tip than the signed dry run: tip changed: signed 5b1e6fd3a7c0, now 111111111111")])
|
||||||
|
self.assertNotIn("tree", result)
|
||||||
|
self.assertEqual(self.tree.read_text(encoding="utf-8"), before)
|
||||||
|
|
||||||
|
def test_an_item_the_apply_keeps_is_failed_with_pkgmgrs_skip_reason(self):
|
||||||
|
kept = self.entry(skip="kept: local upstream of stack", actions=[])
|
||||||
|
(self.fake.parent / "apply.json").write_text(json.dumps([kept]), encoding="utf-8")
|
||||||
|
result, _ = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("failed", "pkgmgr branch cleanup --local keeps it: kept: local upstream of stack")])
|
||||||
|
self.assertNotIn("tree", result)
|
||||||
|
|
||||||
|
def git(self, repo: Path, *args: str) -> str:
|
||||||
|
return subprocess.run(["git", *args], cwd=repo, env={**os.environ, **GIT_ENV}, check=True, capture_output=True, text=True).stdout.strip()
|
||||||
|
|
||||||
|
def scratch(self, *branches: str) -> tuple[Path, str, list[dict], list[dict]]:
|
||||||
|
"""Args:
|
||||||
|
branches: the branches the job signs and the dry run plans, each
|
||||||
|
at the first commit.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
A scratch repository the tree file names, holding only main and
|
||||||
|
feature/two of them, that commit, the signed items and the plan.
|
||||||
|
"""
|
||||||
|
repo = self.repo.parent / "scratch"
|
||||||
|
self.git(self.repo.parent, "init", "-q", "-b", "main", str(repo))
|
||||||
|
self.git(repo, "commit", "-q", "--allow-empty", "-m", "one")
|
||||||
|
tip = self.git(repo, "rev-parse", "HEAD")
|
||||||
|
if "feature/two" in branches:
|
||||||
|
self.git(repo, "branch", "feature/two")
|
||||||
|
self.tree.write_text(json.dumps({"delete_ready": True, "repos": [{"path": str(repo), "known": True}]}), encoding="utf-8")
|
||||||
|
(self.fake.parent / "next-tree.json").write_text(self.tree.read_text(encoding="utf-8"), encoding="utf-8")
|
||||||
|
items = [{"repository": str(repo), "branch": name, "worktree": "", "tip": tip} for name in branches]
|
||||||
|
return repo, tip, items, [self.entry(branch=name, worktree=None, local_sha=tip, head=tip) for name in branches]
|
||||||
|
|
||||||
|
def test_after_an_interrupted_apply_a_gone_branch_is_gone_never_done_and_one_at_its_signed_tip_failed(self):
|
||||||
|
repo, _, items, plan = self.scratch("feature/done")
|
||||||
|
self.git(repo, "branch", "feature/done")
|
||||||
|
(self.fake.parent / "killed").write_text("", encoding="utf-8")
|
||||||
|
result, _ = self.run_job(items, self.proof(items, self.signer), plan)
|
||||||
|
self.assertEqual([entry["status"] for entry in result["items"]], ["failed"])
|
||||||
|
self.assertNotIn("tree", result)
|
||||||
|
self.git(repo, "branch", "-D", "feature/done")
|
||||||
|
result, _ = self.run_job(items, self.proof(items, self.signer), plan)
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("gone", "gone after an interrupted apply")])
|
||||||
|
self.assertEqual(result["tree"], {"code": 0, "error": ""})
|
||||||
|
|
||||||
|
def test_a_branch_gone_although_pkgmgr_answered_without_deleting_it_is_gone_outside_this_run_never_done(self):
|
||||||
|
_, tip, items, plan = self.scratch("feature/done")
|
||||||
|
failed = f"failed: Git command failed: git update-ref --no-deref -d refs/heads/feature/done {tip}"
|
||||||
|
cases = (("update-ref failed", "outcome.txt", failed), ("unknown at the apply", "unknown.json", json.dumps({"apply": ["feature/done"]})))
|
||||||
|
for name, knob, content in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
(self.fake.parent / knob).write_text(content, encoding="utf-8")
|
||||||
|
result, calls = self.run_job(items, self.proof(items, self.signer), plan)
|
||||||
|
(self.fake.parent / knob).unlink()
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("gone", "deleted outside this run")])
|
||||||
|
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"], ["branch", "tree", "--format"]])
|
||||||
|
self.assertEqual(result["tree"], {"code": 0, "error": ""})
|
||||||
|
|
||||||
|
def test_a_branch_that_vanished_is_gone_and_its_repository_runs_once_more_without_it(self):
|
||||||
|
_, _, items, plan = self.scratch("feature/done", "feature/two")
|
||||||
|
unknown = self.fake.parent / "unknown.json"
|
||||||
|
both = ["--local", "--json", "--branch", "feature/done", "--branch", "feature/two"]
|
||||||
|
two = ["--local", "--json", "--branch", "feature/two"]
|
||||||
|
rewrite = ["branch", "tree", "--format", "json", "--output", str(self.tree)]
|
||||||
|
cases = (
|
||||||
|
("before the apply", {"apply": ["feature/done"]}, [("gone", "deleted outside this run"), ("done", "")], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both], ["branch", "cleanup", "--apply", *two], rewrite]),
|
||||||
|
("before the dry run", {"plan": ["feature/done"]}, [("gone", "deleted outside this run"), ("done", "")], [["branch", "cleanup", *both], ["branch", "cleanup", *two], ["branch", "cleanup", "--apply", *two], rewrite]),
|
||||||
|
("once only, never a present branch", {"apply": ["feature/done", "feature/two"]}, [("gone", "deleted outside this run"), ("failed", "Unknown --branch: feature/two")], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both], ["branch", "cleanup", "--apply", *two], rewrite]),
|
||||||
|
)
|
||||||
|
for name, vanished, outcomes, argvs in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
unknown.write_text(json.dumps(vanished), encoding="utf-8")
|
||||||
|
result, calls = self.run_job(items, self.proof(items, self.signer), plan)
|
||||||
|
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], outcomes)
|
||||||
|
self.assertEqual([call["argv"] for call in calls], argvs)
|
||||||
|
|
||||||
|
def test_a_job_plans_and_applies_each_repository_once_for_all_its_items(self):
|
||||||
|
two = {**self.item, "branch": "feature/two", "worktree": ""}
|
||||||
|
items = [self.item, two]
|
||||||
|
(self.fake.parent / "next-tree.json").write_text(self.tree.read_text(encoding="utf-8"), encoding="utf-8")
|
||||||
|
result, calls = self.run_job(items, self.proof(items, self.signer), [self.entry(), self.entry(branch="feature/two", worktree=None)])
|
||||||
|
self.assertEqual([(entry["branch"], entry["status"], entry["reason"]) for entry in result["items"]], [("feature/done", "done", ""), ("feature/two", "done", "")])
|
||||||
|
both = ["--local", "--json", "--branch", "feature/done", "--branch", "feature/two"]
|
||||||
|
self.assertEqual([call["argv"] for call in calls][:2], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both]])
|
||||||
|
result, calls = self.run_job(items, self.proof(items, self.signer), [self.entry(local_sha=OTHER_TIP), self.entry(branch="feature/two", worktree=None)])
|
||||||
|
self.assertEqual([(entry["branch"], entry["status"]) for entry in result["items"]], [("feature/done", "refused"), ("feature/two", "done")], result["items"])
|
||||||
|
self.assertIn("tip changed", result["items"][0]["reason"])
|
||||||
|
self.assertEqual([call["argv"] for call in calls][:2], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", "--local", "--json", "--branch", "feature/two"]])
|
||||||
|
|
||||||
def test_a_signature_deletes_once_and_a_malformed_job_nothing(self):
|
def test_a_signature_deletes_once_and_a_malformed_job_nothing(self):
|
||||||
proof = self.proof([self.item], self.signer)
|
proof = self.proof([self.item], self.signer)
|
||||||
self.assertEqual(self.run_job([self.item], proof, [self.entry()])[0]["items"][0]["status"], "done")
|
self.assertEqual(self.run_job([self.item], proof, [self.entry()])[0]["items"][0]["status"], "done")
|
||||||
@@ -322,21 +610,91 @@ class TestInstalledPkgmgrContract(unittest.TestCase):
|
|||||||
self.git(origin, "fetch", "-q", str(repo), "main:main")
|
self.git(origin, "fetch", "-q", str(repo), "main:main")
|
||||||
self.git(repo, "fetch", "-q", "origin")
|
self.git(repo, "fetch", "-q", "origin")
|
||||||
self.git(repo, "remote", "set-head", "origin", "main")
|
self.git(repo, "remote", "set-head", "origin", "main")
|
||||||
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv("side", False), str(repo))
|
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv(["side"], False), str(repo), host.TIMEOUT)
|
||||||
self.assertEqual(code, 0, stderr)
|
self.assertEqual(code, 0, stderr)
|
||||||
plan = host.plan_items(stdout)
|
plan = host.plan_items(stdout)
|
||||||
self.assertIsNotNone(plan, stdout[:500])
|
self.assertIsNotNone(plan, stdout[:500])
|
||||||
self.assertEqual([(entry["branch"], entry["state"], entry["merged_by"], entry["merged_into"], entry["skip"], entry["actions"]) for entry in plan], [("side", "merged-clean", "ancestor", "origin/main", None, ["local-branch"])])
|
self.assertEqual([(entry["branch"], entry["state"], entry["merged_by"], entry["merged_into"], entry["skip"], entry["actions"]) for entry in plan], [("side", "merged-clean", "ancestor", "refs/remotes/origin/main", None, ["local-branch"])])
|
||||||
self.assertIsNone(host.mismatch(plan[0], {"tip": tip, "worktree": ""}))
|
self.assertIsNone(host.mismatch(plan[0], {"tip": tip, "worktree": ""}))
|
||||||
self.assertTrue(signing.token_free(plan[0], "main"))
|
self.assertTrue(signing.token_free(plan[0], "main"))
|
||||||
|
shown = tree.branch(plan[0], str(repo), "main", plan[0], "", "")
|
||||||
|
self.assertEqual((shown["cleanup"], shown["kept"]), (signing.cleanup_item(str(repo), "side", "", tip), ""))
|
||||||
self.assertEqual(self.git(repo, "rev-parse", "refs/heads/side"), tip)
|
self.assertEqual(self.git(repo, "rev-parse", "refs/heads/side"), tip)
|
||||||
held = base / "held"
|
held = base / "held"
|
||||||
self.git(repo, "worktree", "add", "-q", str(held), "side")
|
self.git(repo, "worktree", "add", "-q", str(held), "side")
|
||||||
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv("side", False), str(repo))
|
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv(["side"], False), str(repo), host.TIMEOUT)
|
||||||
self.assertEqual(code, 0, stderr)
|
self.assertEqual(code, 0, stderr)
|
||||||
plan = host.plan_items(stdout)
|
plan = host.plan_items(stdout)
|
||||||
self.assertEqual([(entry["worktree"], entry["actions"], bool(entry["skip"])) for entry in plan], [(os.path.realpath(held), [], True)], plan)
|
self.assertEqual([(entry["worktree"], entry["actions"], bool(entry["skip"])) for entry in plan], [(os.path.realpath(held), [], True)], plan)
|
||||||
self.assertIn("gc skip reason", host.mismatch(plan[0], {"tip": tip, "worktree": os.path.realpath(held)}))
|
self.assertIn("pkgmgr branch cleanup --local keeps it", host.mismatch(plan[0], {"tip": tip, "worktree": os.path.realpath(held)}))
|
||||||
|
|
||||||
|
def test_the_hub_keeps_the_local_upstream_of_a_merged_branch_exactly_as_the_hosts_dry_run_of_it_alone(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
base = Path(tmp)
|
||||||
|
origin, repo = base / "origin.git", base / "repo"
|
||||||
|
self.git(base, "init", "-q", "--bare", "-b", "main", str(origin))
|
||||||
|
self.git(base, "init", "-q", "-b", "main", str(repo))
|
||||||
|
self.git(repo, "remote", "add", "origin", origin.as_uri())
|
||||||
|
self.git(repo, "commit", "-q", "--allow-empty", "-m", "one")
|
||||||
|
self.git(repo, "checkout", "-q", "-b", "base")
|
||||||
|
self.git(repo, "commit", "-q", "--allow-empty", "-m", "base")
|
||||||
|
tip = self.git(repo, "rev-parse", "HEAD")
|
||||||
|
self.git(repo, "checkout", "-q", "main")
|
||||||
|
self.git(repo, "merge", "-q", "--no-ff", "-m", "merge base", "base")
|
||||||
|
self.git(repo, "branch", "stack", "base")
|
||||||
|
self.git(repo, "branch", "-q", "--set-upstream-to=base", "stack")
|
||||||
|
self.git(origin, "fetch", "-q", str(repo), "main:main")
|
||||||
|
self.git(repo, "fetch", "-q", "origin")
|
||||||
|
self.git(repo, "remote", "set-head", "origin", "main")
|
||||||
|
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), ["branch", "cleanup", "--local", "--json"], str(repo), host.TIMEOUT)
|
||||||
|
self.assertEqual(code, 0, stderr)
|
||||||
|
planned = base / "local-plan.json"
|
||||||
|
planned.write_text(stdout, encoding="utf-8")
|
||||||
|
hub = {branch: tree.kept_by(item, tip, child, "") for (_, branch), (item, child) in tree.local_plan(planned)[0].items() if branch in ("base", "stack")}
|
||||||
|
refusals = {}
|
||||||
|
for name in ("base", "stack"):
|
||||||
|
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv([name], False), str(repo), host.TIMEOUT)
|
||||||
|
self.assertEqual(code, 0, stderr)
|
||||||
|
refusals[name] = host.mismatch(host.plan_items(stdout)[0], {"tip": tip, "worktree": ""})
|
||||||
|
self.assertEqual(refusals, {"base": "pkgmgr branch cleanup --local keeps it: kept: local upstream of stack", "stack": None})
|
||||||
|
self.assertEqual(hub, {"base": f"{refusals['base']}; use branch gc (kvb-git-gc, pkgmgr gc)", "stack": ""})
|
||||||
|
|
||||||
|
def test_the_installed_pkgmgr_plans_the_recorded_fixture_exactly(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
repo, plan = capture_local_plan(Path(tmp), Path(INSTALLED_PKGMGR))
|
||||||
|
self.assertEqual(plan["repos"], recorded_plan(str(repo))["repos"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestRecordedLocalPlan(HubCase):
|
||||||
|
def setUp(self):
|
||||||
|
super().setUp()
|
||||||
|
self.repo = "/home/operator-example-01/repos/demo"
|
||||||
|
self.items = {entry["branch"]: entry for entry in recorded_plan(self.repo)["repos"][0]["items"]}
|
||||||
|
|
||||||
|
def test_the_runner_and_the_tree_agree_on_every_recorded_branch(self):
|
||||||
|
expected = {"side": "", "topic": "", "squashed": f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}"}
|
||||||
|
self.assertEqual({name: self.items[name]["merged_by"] for name in expected}, {"side": "ancestor", "topic": "patch", "squashed": "patch"})
|
||||||
|
for name, refused in expected.items():
|
||||||
|
with self.subTest(branch=name):
|
||||||
|
entry = self.items[name]
|
||||||
|
self.assertEqual(host.mismatch(entry, {"tip": entry["local_sha"], "worktree": ""}), refused or None)
|
||||||
|
shown = tree.branch(entry, self.repo, "main", entry, "", "")
|
||||||
|
offered = signing.cleanup_item(self.repo, name, "", entry["local_sha"])
|
||||||
|
self.assertEqual((shown["cleanup"], shown["kept"]), (None, f"{refused}; use branch gc (kvb-git-gc, pkgmgr gc)") if refused else (offered, ""))
|
||||||
|
|
||||||
|
def test_the_tree_offers_the_branch_whose_local_tip_is_an_ancestor_although_its_remote_tip_was_cherry_picked(self):
|
||||||
|
plan = recorded_plan(self.repo)
|
||||||
|
tree_data = {"generated_at": "2026-11-14T12:00:00+00:00", "delete_ready": True, "delete_reason": "", "repos": [{"path": self.repo, "remote": None, "default": "main", "missing": False, "known": True, "forge": "live", "scanned_at": "2026-11-14T12:00:00+00:00", "warnings": [], "items": [{**entry, "flow": None} for entry in plan["repos"][0]["items"]]}]}
|
||||||
|
self.tree_file.write_text(json.dumps(tree_data), encoding="utf-8")
|
||||||
|
self.tree_file.with_name("local-plan.json").write_text(json.dumps(plan), encoding="utf-8")
|
||||||
|
status, _, body = self.get("/api/tree")
|
||||||
|
self.assertEqual(status, 200, body)
|
||||||
|
branches = {entry["name"]: entry for entry in json.loads(body)["repos"][0]["branches"]}
|
||||||
|
self.assertEqual({name: (entry["cleanup"] is not None, entry["kept"]) for name, entry in branches.items()}, {
|
||||||
|
"main": (False, ""), "side": (True, ""), "topic": (True, ""), "squashed": (False, f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}; use branch gc (kvb-git-gc, pkgmgr gc)"),
|
||||||
|
})
|
||||||
|
topic = self.items["topic"]
|
||||||
|
self.assertEqual(self.post_json("/api/sign", {"items": [signing.cleanup_item(self.repo, "topic", "", topic["local_sha"])]}, self.guard(self.host))[0], 200)
|
||||||
|
|
||||||
|
|
||||||
class TestHostCleanupGitConfig(unittest.TestCase):
|
class TestHostCleanupGitConfig(unittest.TestCase):
|
||||||
@@ -361,12 +719,45 @@ class TestHostCleanupGitConfig(unittest.TestCase):
|
|||||||
fake = base / "pkgmgr"
|
fake = base / "pkgmgr"
|
||||||
fake.write_text(f"#!{sys.executable}\nimport subprocess\nsubprocess.run(['git', 'branch', '-D', '--', 'doomed'], check=True)\n", encoding="utf-8")
|
fake.write_text(f"#!{sys.executable}\nimport subprocess\nsubprocess.run(['git', 'branch', '-D', '--', 'doomed'], check=True)\n", encoding="utf-8")
|
||||||
fake.chmod(0o755)
|
fake.chmod(0o755)
|
||||||
code, _, error = host.run(fake, ["branch", "cleanup", "--apply"], str(repo))
|
code, _, error = host.run(fake, ["branch", "cleanup", "--apply"], str(repo), host.TIMEOUT)
|
||||||
self.assertEqual(code, 0, error)
|
self.assertEqual(code, 0, error)
|
||||||
self.assertFalse(marker.exists())
|
self.assertFalse(marker.exists())
|
||||||
branches = subprocess.run(["git", "branch", "--format=%(refname:short)"], cwd=repo, capture_output=True, text=True, check=True).stdout.split()
|
branches = subprocess.run(["git", "branch", "--format=%(refname:short)"], cwd=repo, capture_output=True, text=True, check=True).stdout.split()
|
||||||
self.assertNotIn("doomed", branches)
|
self.assertNotIn("doomed", branches)
|
||||||
|
|
||||||
|
|
||||||
|
RESULT_VIEW = """
|
||||||
|
const fs = require("fs");
|
||||||
|
const vm = require("vm");
|
||||||
|
const context = {h: (tag, attrs, ...kids) => ({tag, attrs: attrs || {}, kids}), named: (label) => ({"aria-label": label}), icon: (name) => name, pill: (label, tone) => ({pill: label, tone})};
|
||||||
|
vm.runInNewContext(`${fs.readFileSync(process.argv[1], "utf8")}\\n${fs.readFileSync(process.argv[2], "utf8")}\\nglobalThis.probe = {resultChip, showJob, RESULT_LABELS};`, context);
|
||||||
|
const statuses = ["done", "gone", "failed", "refused"];
|
||||||
|
const chips = statuses.map((status) => {
|
||||||
|
const chip = context.probe.resultChip({status, reason: "why"});
|
||||||
|
return [status, chip.attrs["data-tone"], chip.attrs["aria-label"], chip.kids[0]];
|
||||||
|
});
|
||||||
|
const outcome = {replaceChildren: (list) => { outcome.list = list; }};
|
||||||
|
const shown = context.probe.showJob({items: statuses.map((status) => ({branch: status, status, reason: ""}))}, outcome, "branch", context.probe.RESULT_LABELS);
|
||||||
|
console.log(JSON.stringify({chips, pills: outcome.list.kids[0].map((row) => [row.attrs["data-result"], row.kids[0].kids[1]]), good: shown.good}));
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
@unittest.skipUnless(shutil.which("node"), "no node")
|
||||||
|
class TestCleanupResultView(unittest.TestCase):
|
||||||
|
def test_the_tree_and_the_delete_sheet_show_a_gone_branch_apart_from_deleted_and_failed(self):
|
||||||
|
pages = [str(HUB / "templates" / name) for name in ("page-tree.js", "page-cleanup.js")]
|
||||||
|
done = subprocess.run(["node", "-e", RESULT_VIEW, *pages], capture_output=True, text=True, timeout=60, check=False)
|
||||||
|
self.assertEqual(done.returncode, 0, done.stderr[-2000:])
|
||||||
|
shown = json.loads(done.stdout)
|
||||||
|
self.assertEqual(shown["chips"], [
|
||||||
|
["done", "ok", "deleted: why", "bi-check2-all"],
|
||||||
|
["gone", "warn", "gone: why", "bi-question-octagon"],
|
||||||
|
["failed", "bad", "failed: why", "bi-exclamation-triangle"],
|
||||||
|
["refused", "bad", "refused: why", "bi-x-octagon"],
|
||||||
|
])
|
||||||
|
self.assertEqual(shown["pills"], [["done", {"pill": "deleted", "tone": "ok"}], ["gone", {"pill": "gone", "tone": "warn"}], ["failed", {"pill": "failed", "tone": "bad"}], ["refused", {"pill": "refused", "tone": "bad"}]])
|
||||||
|
self.assertFalse(shown["good"])
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
@@ -14,6 +14,7 @@ from pathlib import Path
|
|||||||
|
|
||||||
from hub_case import HubCase
|
from hub_case import HubCase
|
||||||
from hub_support import GIT_ENV, HUB, ORIGIN, REPO_ROOT, key, signing, verifier
|
from hub_support import GIT_ENV, HUB, ORIGIN, REPO_ROOT, key, signing, verifier
|
||||||
|
from test_hub_cleanup import INSTALLED_PKGMGR
|
||||||
|
|
||||||
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_token_runner", HUB / "client" / "cleanup.py")
|
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_token_runner", HUB / "client" / "cleanup.py")
|
||||||
host = importlib.util.module_from_spec(CLEANUP_SPEC)
|
host = importlib.util.module_from_spec(CLEANUP_SPEC)
|
||||||
@@ -23,7 +24,7 @@ operator_script = importlib.util.module_from_spec(OPERATOR_SPEC)
|
|||||||
OPERATOR_SPEC.loader.exec_module(operator_script)
|
OPERATOR_SPEC.loader.exec_module(operator_script)
|
||||||
KEPT_TIP = "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"
|
KEPT_TIP = "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"
|
||||||
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
|
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
|
||||||
# Param: format with python=<interpreter>; plans one branch of the real repository in its cwd as pkgmgr 2.4.0 `branch cleanup` does without a GitHub token: argv parsed as branch_cmd.py does (exit 2 on any other flag), merged_into the remote HEAD ref (gc.py symbolic-ref --short), an origin on github.com skips every item with "forge unavailable" (gc.py _skip), actions local-branch plus remote-branch:<remote> (gc.py _actions), --apply deletes only an item without skip (Item.eligible) and the output is gc.py _render's {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}.
|
# Param: format with python=<interpreter>; plans one branch of the real repository in its cwd as pkgmgr 2.5.0 `branch cleanup --local` does, which reads no forge token: argv parsed as branch_cmd.py does (exit 2 on any other flag), default and merged_into the full remote HEAD ref (refs/remotes/origin/main), no forge skip on a github.com origin, actions only local-branch while the remote branch stays listed under remotes, --apply deletes only a merged item and the output is gc.py _render's {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}.
|
||||||
SCRATCH_PKGMGR = """#!{python}
|
SCRATCH_PKGMGR = """#!{python}
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
@@ -36,7 +37,7 @@ tree = branch.add_parser("tree")
|
|||||||
tree.add_argument("--format", choices=("json", "yaml"), default="json")
|
tree.add_argument("--format", choices=("json", "yaml"), default="json")
|
||||||
tree.add_argument("--output")
|
tree.add_argument("--output")
|
||||||
cleanup = branch.add_parser("cleanup")
|
cleanup = branch.add_parser("cleanup")
|
||||||
for flag in ("--apply", "--json", "--all", "--sizes"):
|
for flag in ("--apply", "--json", "--local", "--all", "--sizes"):
|
||||||
cleanup.add_argument(flag, action="store_true")
|
cleanup.add_argument(flag, action="store_true")
|
||||||
for option in ("--branch", "--exclude", "--merged-into"):
|
for option in ("--branch", "--exclude", "--merged-into"):
|
||||||
cleanup.add_argument(option, action="append", default=[])
|
cleanup.add_argument(option, action="append", default=[])
|
||||||
@@ -50,20 +51,25 @@ def git(*argv):
|
|||||||
return subprocess.run(["git", *argv], capture_output=True, text=True, check=False)
|
return subprocess.run(["git", *argv], capture_output=True, text=True, check=False)
|
||||||
|
|
||||||
|
|
||||||
name = args.branch[0]
|
default = git("symbolic-ref", "refs/remotes/origin/HEAD").stdout.strip()
|
||||||
tip = git("rev-parse", "--verify", "refs/heads/" + name).stdout.strip()
|
|
||||||
default = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD").stdout.strip()
|
|
||||||
merged = bool(tip) and git("merge-base", "--is-ancestor", tip, default).returncode == 0
|
|
||||||
listed = git("for-each-ref", "refs/remotes", "--format=%(refname:lstrip=2) %(objectname)").stdout.split("\\n")
|
listed = git("for-each-ref", "refs/remotes", "--format=%(refname:lstrip=2) %(objectname)").stdout.split("\\n")
|
||||||
remotes = {{line.split("/", 1)[0]: line.split(" ")[1] for line in listed if line and line.split(" ")[0].split("/", 1)[1] == name}}
|
|
||||||
skip = "forge unavailable" if "github.com" in git("remote", "get-url", "origin").stdout else None
|
|
||||||
actions = ["local-branch", *("remote-branch:" + remote for remote in remotes)] if merged else []
|
def planned(name):
|
||||||
entry = {{"branch": name, "worktree": None, "local_sha": tip, "head": None, "state": "merged-clean" if merged else "unmerged", "skip": skip,
|
tip = git("rev-parse", "--verify", "refs/heads/" + name).stdout.strip()
|
||||||
"merged_by": "ancestor" if merged else None, "merged_into": default if merged else None, "remotes": remotes, "actions": actions, "result": None}}
|
merged = bool(tip) and git("merge-base", "--is-ancestor", tip, default).returncode == 0
|
||||||
if args.apply and skip is None:
|
remotes = {{line.split("/", 1)[0]: line.split(" ")[1] for line in listed if line and line.split(" ")[0].split("/", 1)[1] == name}}
|
||||||
git("branch", "-D", "--", name)
|
entry = {{"branch": name, "worktree": None, "local_sha": tip, "head": None, "state": "merged-clean" if merged else "unmerged", "skip": None,
|
||||||
entry["result"] = "done"
|
"merged_by": "ancestor" if merged else None, "merged_into": default if merged else None, "remotes": remotes, "result": None}}
|
||||||
print(json.dumps({{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": default, "merged_into": [], "warnings": [], "items": [entry]}}]}}))
|
entry["actions"] = ["local-branch"] if merged else []
|
||||||
|
if args.apply and merged:
|
||||||
|
git("branch", "-D", "--", name)
|
||||||
|
entry["result"] = "done"
|
||||||
|
return entry
|
||||||
|
|
||||||
|
|
||||||
|
items = [planned(name) for name in args.branch]
|
||||||
|
print(json.dumps({{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": default, "merged_into": [], "warnings": [], "items": items}}]}}))
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
@@ -75,8 +81,9 @@ class TestHubCleanupToken(HubCase):
|
|||||||
kept = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/kept")
|
kept = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/kept")
|
||||||
kept["skip"] = kept_skip
|
kept["skip"] = kept_skip
|
||||||
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
|
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
|
||||||
done.update(worktree=None, worktree_state=None, actions=["local-branch"], compose_projects=[], images=[])
|
done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch:origin"], compose_projects=[], images=[], merged_by="ancestor", remotes={"origin": DONE_TIP})
|
||||||
self.tree_file.write_text(json.dumps(data), encoding="utf-8")
|
self.tree_file.write_text(json.dumps(data), encoding="utf-8")
|
||||||
|
self.write_local_plan(data, {})
|
||||||
return root
|
return root
|
||||||
|
|
||||||
def items(self, root: Path) -> tuple[dict, dict]:
|
def items(self, root: Path) -> tuple[dict, dict]:
|
||||||
@@ -96,10 +103,14 @@ class TestHubCleanupToken(HubCase):
|
|||||||
self.assertEqual({name: entry["token_free"] for name, entry in branches.items()}, {"feature": False, "feature/done": False, "feature/kept": True, "main": False})
|
self.assertEqual({name: entry["token_free"] for name, entry in branches.items()}, {"feature": False, "feature/done": False, "feature/kept": True, "main": False})
|
||||||
entry = {"merged_by": "ancestor", "merged_into": "main", "remotes": {}}
|
entry = {"merged_by": "ancestor", "merged_into": "main", "remotes": {}}
|
||||||
cases = (("ancestry, no remote", entry, "main", True), ("pull request", {**entry, "merged_by": "#12"}, "main", False),
|
cases = (("ancestry, no remote", entry, "main", True), ("pull request", {**entry, "merged_by": "#12"}, "main", False),
|
||||||
|
("pull request as pkgmgr names it", {**entry, "merged_by": "pr#12"}, "main", False), ("patch", {**entry, "merged_by": "patch"}, "main", False),
|
||||||
("remote branch", {**entry, "remotes": {"origin": "a" * 40}}, "main", False), ("other line", {**entry, "merged_into": "store-i18n"}, "main", False),
|
("remote branch", {**entry, "remotes": {"origin": "a" * 40}}, "main", False), ("other line", {**entry, "merged_into": "store-i18n"}, "main", False),
|
||||||
("no default", entry, None, False), ("line merge", {**entry, "merged_by": "line:main"}, "main", False),
|
("no default", entry, None, False), ("line merge", {**entry, "merged_by": "line:main"}, "main", False),
|
||||||
("pkgmgr names the remote HEAD", {**entry, "merged_into": "origin/main"}, "main", True), ("other remote line", {**entry, "merged_into": "origin/store-i18n"}, "main", False),
|
("pkgmgr names the remote HEAD", {**entry, "merged_into": "origin/main"}, "main", True), ("other remote line", {**entry, "merged_into": "origin/store-i18n"}, "main", False),
|
||||||
("nested name", {**entry, "merged_into": "origin/feature/main"}, "main", False), ("no merged_into", {**entry, "merged_into": None}, "main", False))
|
("nested name", {**entry, "merged_into": "origin/feature/main"}, "main", False), ("no merged_into", {**entry, "merged_into": None}, "main", False),
|
||||||
|
("--local names the full remote ref", {**entry, "merged_into": "refs/remotes/origin/main"}, "main", True),
|
||||||
|
("full ref of another line", {**entry, "merged_into": "refs/remotes/origin/store-i18n"}, "main", False),
|
||||||
|
("full local ref", {**entry, "merged_into": "refs/heads/main"}, "main", False))
|
||||||
for name, candidate, default, free in cases:
|
for name, candidate, default, free in cases:
|
||||||
with self.subTest(name=name):
|
with self.subTest(name=name):
|
||||||
self.assertEqual(signing.token_free(candidate, default), free)
|
self.assertEqual(signing.token_free(candidate, default), free)
|
||||||
@@ -123,19 +134,25 @@ class TestHubCleanupToken(HubCase):
|
|||||||
status, _, body = self.get("/api/tree")
|
status, _, body = self.get("/api/tree")
|
||||||
self.assertEqual(status, 200, body)
|
self.assertEqual(status, 200, body)
|
||||||
repos = {repo["name"]: {entry["name"]: entry for entry in repo["branches"]} for repo in json.loads(body)["repos"]}
|
repos = {repo["name"]: {entry["name"]: entry for entry in repo["branches"]} for repo in json.loads(body)["repos"]}
|
||||||
shown = {name: {branch: (entry["token_free"], entry["cleanup"] is not None, entry["skip"]) for branch, entry in branches.items()} for name, branches in repos.items()}
|
shown = {name: {branch: (entry["token_free"], entry["cleanup"] is not None, entry["skip"], bool(entry["kept"])) for branch, entry in branches.items()} for name, branches in repos.items()}
|
||||||
self.assertEqual(shown, {
|
self.assertEqual(shown, {
|
||||||
"github.com/anon/web": {"chore/ancestor-local": (True, False, "forge unavailable"), "fix/remote-merged": (False, False, "forge unavailable"), "feature/open": (False, False, "forge unavailable"), "main": (False, False, None)},
|
"github.com/anon/web": {"chore/ancestor-local": (True, False, "forge unavailable", False), "fix/remote-merged": (False, False, "forge unavailable", False), "feature/open": (False, False, "forge unavailable", False), "main": (False, False, None, False)},
|
||||||
"git.example.org/anon/notes": {"topic/ancestor-local": (True, True, None), "topic/patch-local": (False, True, None), "master": (False, False, None)},
|
"git.example.org/anon/notes": {"topic/ancestor-local": (True, True, None, False), "topic/patch-local": (False, False, None, True), "master": (False, False, None, False)},
|
||||||
})
|
})
|
||||||
notes = repos["git.example.org/anon/notes"]
|
notes = repos["git.example.org/anon/notes"]
|
||||||
free, patch = notes["topic/ancestor-local"]["cleanup"], notes["topic/patch-local"]["cleanup"]
|
free = notes["topic/ancestor-local"]["cleanup"]
|
||||||
self.assertEqual(free, signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/ancestor-local", "", "5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f"))
|
self.assertEqual(free, signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/ancestor-local", "", "5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f"))
|
||||||
status, _, body = self.post_json("/api/sign", {"items": [free]}, self.guard(self.host))
|
status, _, body = self.post_json("/api/sign", {"items": [free]}, self.guard(self.host))
|
||||||
self.assertEqual(status, 200, body)
|
self.assertEqual(status, 200, body)
|
||||||
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(free)])
|
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(free)])
|
||||||
status, _, body = self.post_json("/api/sign", {"items": [patch]}, self.guard(self.host))
|
self.assertEqual(notes["topic/patch-local"]["plan"][1], "pkgmgr branch cleanup --local keeps it: merged by patch, tip is no ancestor of refs/remotes/origin/master; use branch gc (kvb-git-gc, pkgmgr gc)")
|
||||||
self.assertEqual((status, body.decode()), (400, "delete blocked: no GitHub token in the Secret Service\n"))
|
patch = signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/patch-local", "", "6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a")
|
||||||
|
data = json.loads(self.tree_file.read_text(encoding="utf-8"))
|
||||||
|
for ready in (False, True):
|
||||||
|
with self.subTest(delete_ready=ready):
|
||||||
|
self.tree_file.write_text(json.dumps({**data, "delete_ready": ready}), encoding="utf-8")
|
||||||
|
status, _, body = self.post_json("/api/sign", {"items": [patch]}, self.guard(self.host))
|
||||||
|
self.assertEqual((status, body.decode()), (400, f"topic/patch-local in {root}/repos/git.example.org/anon/notes is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"))
|
||||||
|
|
||||||
def test_with_the_token_every_merged_clean_branch_without_skip_reason_is_offered(self):
|
def test_with_the_token_every_merged_clean_branch_without_skip_reason_is_offered(self):
|
||||||
kept, done = self.items(self.tree(True, None))
|
kept, done = self.items(self.tree(True, None))
|
||||||
@@ -154,6 +171,7 @@ class TestHubCleanupToken(HubCase):
|
|||||||
command = args[args.index("--command") + 1]
|
command = args[args.index("--command") + 1]
|
||||||
self.assertEqual(shlex.split(command), ["env", "secret-tool", "store", "--label=pkgmgr-github", "service", "pkgmgr:github", "username", "github.com|-", "</dev/tty"])
|
self.assertEqual(shlex.split(command), ["env", "secret-tool", "store", "--label=pkgmgr-github", "service", "pkgmgr:github", "username", "github.com|-", "</dev/tty"])
|
||||||
self.assertEqual(args[args.index("--check") + 1], "manual")
|
self.assertEqual(args[args.index("--check") + 1], "manual")
|
||||||
|
self.assertEqual(args[args.index("--reason") + 1], "Store the GitHub token in the Secret Service under the key pkgmgr reads; Delete all merged then also deletes the local branch of a merged branch that has a remote branch; never the remote branch.")
|
||||||
self.assertIsNone(operator_script.unrunnable(command, "/"))
|
self.assertIsNone(operator_script.unrunnable(command, "/"))
|
||||||
self.assertEqual(self.post_json("/api/cleanup/token", {"extra": 1}, self.guard(self.host))[0], 400)
|
self.assertEqual(self.post_json("/api/cleanup/token", {"extra": 1}, self.guard(self.host))[0], 400)
|
||||||
self.assertEqual(self.post_json("/api/cleanup/token", {}, {"Host": self.host})[0], 403)
|
self.assertEqual(self.post_json("/api/cleanup/token", {}, {"Host": self.host})[0], 403)
|
||||||
@@ -168,13 +186,13 @@ class TestHostCleanupWithoutToken(unittest.TestCase):
|
|||||||
self.repo = base / "repo"
|
self.repo = base / "repo"
|
||||||
self.git("init", "-q", "-b", "main", str(self.repo), cwd=base)
|
self.git("init", "-q", "-b", "main", str(self.repo), cwd=base)
|
||||||
self.git("commit", "-q", "--allow-empty", "-m", "one")
|
self.git("commit", "-q", "--allow-empty", "-m", "one")
|
||||||
self.git("branch", "local-merged")
|
for name in ("local-merged", "remote-merged", "squash-candidate"):
|
||||||
self.git("branch", "remote-merged")
|
self.git("checkout", "-q", "-b", name, "main")
|
||||||
self.git("branch", "squash-candidate")
|
(self.repo / f"{name}.txt").write_text(name, encoding="utf-8")
|
||||||
self.git("commit", "-q", "--allow-empty", "-m", "two")
|
self.git("add", f"{name}.txt")
|
||||||
self.git("checkout", "-q", "squash-candidate")
|
self.git("commit", "-q", "-m", name)
|
||||||
self.git("commit", "-q", "--allow-empty", "-m", "side")
|
|
||||||
self.git("checkout", "-q", "main")
|
self.git("checkout", "-q", "main")
|
||||||
|
self.git("merge", "-q", "--no-ff", "-m", "merge", "local-merged", "remote-merged")
|
||||||
self.git("remote", "add", "origin", "ssh://git@git.example.org/anon/repo.git")
|
self.git("remote", "add", "origin", "ssh://git@git.example.org/anon/repo.git")
|
||||||
self.git("update-ref", "refs/remotes/origin/main", self.tip("main"))
|
self.git("update-ref", "refs/remotes/origin/main", self.tip("main"))
|
||||||
self.git("symbolic-ref", "refs/remotes/origin/HEAD", "refs/remotes/origin/main")
|
self.git("symbolic-ref", "refs/remotes/origin/HEAD", "refs/remotes/origin/main")
|
||||||
@@ -213,28 +231,40 @@ class TestHostCleanupWithoutToken(unittest.TestCase):
|
|||||||
result = json.loads((self.results / f"{job_id}.json").read_text(encoding="utf-8"))
|
result = json.loads((self.results / f"{job_id}.json").read_text(encoding="utf-8"))
|
||||||
return {entry["branch"]: entry for entry in result["items"]}
|
return {entry["branch"]: entry for entry in result["items"]}
|
||||||
|
|
||||||
def test_the_executor_deletes_a_local_merged_branch_without_token_and_refuses_a_remote_one(self):
|
@unittest.skipUnless(INSTALLED_PKGMGR, "no pkgmgr installed")
|
||||||
|
def test_the_scratch_model_plans_every_fixture_branch_as_the_installed_pkgmgr_does(self):
|
||||||
|
keys = ("state", "skip", "merged_by", "merged_into", "remotes", "actions", "worktree", "local_sha")
|
||||||
|
cases = (("ssh://git@git.example.org/anon/repo.git", ("local-merged", "remote-merged", "squash-candidate")),
|
||||||
|
("git@github.com:anon/repo.git", ("local-merged",)))
|
||||||
|
for url, names in cases:
|
||||||
|
self.git("remote", "set-url", "origin", url)
|
||||||
|
for name in names:
|
||||||
|
with self.subTest(url=url, name=name):
|
||||||
|
plans = [host.plan_items(host.run(Path(pkgmgr), host.plan_argv([name], False), str(self.repo), host.TIMEOUT)[1]) for pkgmgr in (INSTALLED_PKGMGR, self.fake)]
|
||||||
|
self.assertEqual(*[[{key: entry.get(key) for key in keys} for entry in plan] for plan in plans])
|
||||||
|
|
||||||
|
def test_without_token_the_executor_deletes_a_local_merged_branch_and_refuses_one_with_a_remote_branch(self):
|
||||||
result = self.run_job(["local-merged", "remote-merged"])
|
result = self.run_job(["local-merged", "remote-merged"])
|
||||||
self.assertEqual(result["local-merged"]["status"], "done", result["local-merged"]["reason"])
|
self.assertEqual(result["local-merged"]["status"], "done", result["local-merged"]["reason"])
|
||||||
self.assertEqual(result["remote-merged"]["status"], "refused")
|
self.assertEqual((result["remote-merged"]["status"], result["remote-merged"]["reason"]), ("refused", host.TOKEN_NEEDED))
|
||||||
self.assertIn("a signed delete is local only", result["remote-merged"]["reason"])
|
|
||||||
self.assertEqual(self.branches(), {"main", "remote-merged", "squash-candidate"})
|
self.assertEqual(self.branches(), {"main", "remote-merged", "squash-candidate"})
|
||||||
|
|
||||||
def test_without_token_an_unmerged_squash_candidate_stays_and_with_the_token_a_plan_that_deletes_the_remote_branch_is_refused(self):
|
def test_without_token_an_unmerged_squash_candidate_stays_and_with_the_token_a_remote_merged_branch_loses_only_its_local_branch(self):
|
||||||
result = self.run_job(["squash-candidate"])
|
result = self.run_job(["squash-candidate"])
|
||||||
self.assertEqual(result["squash-candidate"]["status"], "refused")
|
self.assertEqual(result["squash-candidate"]["status"], "refused")
|
||||||
self.assertIn("not merged-clean", result["squash-candidate"]["reason"])
|
self.assertIn("not merged-clean", result["squash-candidate"]["reason"])
|
||||||
data = json.loads(self.tree.read_text(encoding="utf-8"))
|
data = json.loads(self.tree.read_text(encoding="utf-8"))
|
||||||
self.tree.write_text(json.dumps({**data, "delete_ready": True}), encoding="utf-8")
|
self.tree.write_text(json.dumps({**data, "delete_ready": True}), encoding="utf-8")
|
||||||
remote = self.run_job(["remote-merged"])["remote-merged"]
|
remote = self.run_job(["remote-merged"])["remote-merged"]
|
||||||
self.assertEqual((remote["status"], remote["reason"]), ("refused", "the plan also deletes remote-branch:origin; a signed delete is local only"))
|
self.assertEqual((remote["status"], remote["reason"]), ("done", ""))
|
||||||
self.assertEqual(self.branches(), {"main", "local-merged", "remote-merged", "squash-candidate"})
|
self.assertEqual(self.branches(), {"main", "local-merged", "squash-candidate"})
|
||||||
|
self.assertTrue(self.git("rev-parse", "--verify", "refs/remotes/origin/remote-merged"))
|
||||||
|
|
||||||
def test_without_token_pkgmgr_skips_every_item_of_a_github_repository_so_the_executor_refuses_and_keeps_even_the_token_free_branch(self):
|
def test_on_a_github_origin_the_local_merged_branch_is_deleted_without_token(self):
|
||||||
self.git("remote", "set-url", "origin", "git@github.com:anon/repo.git")
|
self.git("remote", "set-url", "origin", "git@github.com:anon/repo.git")
|
||||||
result = self.run_job(["local-merged"])["local-merged"]
|
result = self.run_job(["local-merged"])["local-merged"]
|
||||||
self.assertEqual((result["status"], result["reason"]), ("refused", "gc skip reason: forge unavailable"))
|
self.assertEqual((result["status"], result["reason"]), ("done", ""))
|
||||||
self.assertIn("local-merged", self.branches())
|
self.assertNotIn("local-merged", self.branches())
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -0,0 +1,180 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from hub_case import HubCase
|
||||||
|
from hub_support import GIT_ENV, HUB
|
||||||
|
from test_hub_cleanup import (
|
||||||
|
FAKE_PKGMGR,
|
||||||
|
INSTALLED_PKGMGR,
|
||||||
|
KEPT_PATCH,
|
||||||
|
capture_local_plan,
|
||||||
|
)
|
||||||
|
|
||||||
|
LOCAL_PLAN = HUB / "client" / "local_plan.py"
|
||||||
|
TREE_UNIT = HUB / "systemd" / "kvb-orchestrator-tree.service"
|
||||||
|
|
||||||
|
|
||||||
|
class TestLocalPlan(HubCase):
|
||||||
|
def fake(self) -> Path:
|
||||||
|
fake = self.outside / "bin" / "pkgmgr"
|
||||||
|
fake.parent.mkdir(parents=True)
|
||||||
|
fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8")
|
||||||
|
fake.chmod(0o755)
|
||||||
|
return fake
|
||||||
|
|
||||||
|
def argv(self, pkgmgr: Path, budget: int) -> list[str]:
|
||||||
|
return [sys.executable, str(LOCAL_PLAN), "--tree", str(self.tree_file), "--pkgmgr", str(pkgmgr), "--budget", str(budget)]
|
||||||
|
|
||||||
|
def plan(self, pkgmgr: Path, env: dict[str, str]) -> subprocess.CompletedProcess:
|
||||||
|
return subprocess.run(self.argv(pkgmgr, 240), env=env, capture_output=True, text=True, timeout=300, check=False)
|
||||||
|
|
||||||
|
def repository(self, name: str) -> Path:
|
||||||
|
repo = Path(os.path.realpath(self.outside)) / "repos" / name
|
||||||
|
for folder in ("refs/heads", "refs/remotes/origin", "worktrees/demo-side"):
|
||||||
|
(repo / ".git" / folder).mkdir(parents=True)
|
||||||
|
for path, text in (("HEAD", "ref: refs/heads/main\n"), ("config", "[core]\n"), ("refs/heads/side", "a" * 40 + "\n"), ("worktrees/demo-side/index", "index")):
|
||||||
|
(repo / ".git" / path).write_text(text, encoding="utf-8")
|
||||||
|
return repo
|
||||||
|
|
||||||
|
def calls(self, fake: Path) -> list[list[str]]:
|
||||||
|
log = fake.parent / "calls.jsonl"
|
||||||
|
found = [[json.loads(line)["cwd"], *json.loads(line)["argv"]] for line in log.read_text(encoding="utf-8").splitlines()] if log.exists() else []
|
||||||
|
log.unlink(missing_ok=True)
|
||||||
|
return found
|
||||||
|
|
||||||
|
def written(self) -> dict[str, dict]:
|
||||||
|
data = json.loads(self.tree_file.with_name("local-plan.json").read_text(encoding="utf-8"))
|
||||||
|
return {Path(entry["repo"]).name: entry for entry in data["repos"]}
|
||||||
|
|
||||||
|
def test_it_plans_only_the_known_repositories_whose_refs_or_config_changed_since_their_last_plan(self):
|
||||||
|
fake = self.fake()
|
||||||
|
items = [{"branch": "side", "local_sha": "a" * 40, "state": "merged-clean", "skip": None, "actions": ["local-branch"]}]
|
||||||
|
(fake.parent / "plan.json").write_text(json.dumps(items), encoding="utf-8")
|
||||||
|
repo = self.repository("demo")
|
||||||
|
stranger = self.outside / "repos" / "stranger"
|
||||||
|
stranger.mkdir()
|
||||||
|
self.tree_file.write_text(json.dumps({"repos": [{"path": str(repo), "known": True, "missing": False}, {"path": str(stranger), "known": False, "missing": False}]}), encoding="utf-8")
|
||||||
|
written = self.tree_file.with_name("local-plan.json")
|
||||||
|
|
||||||
|
def run(changes: dict[str, str]) -> tuple[int, list[list[str]], dict]:
|
||||||
|
for name, text in changes.items():
|
||||||
|
(repo / ".git" / name).parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
(repo / ".git" / name).write_text(text, encoding="utf-8")
|
||||||
|
done = self.plan(fake, dict(os.environ))
|
||||||
|
data = json.loads(written.read_text(encoding="utf-8"))
|
||||||
|
self.assertTrue(all(len(entry.pop("fingerprint")) == 64 for entry in data["repos"]))
|
||||||
|
return done.returncode, self.calls(fake), data
|
||||||
|
|
||||||
|
code, calls, plan = run({})
|
||||||
|
self.assertEqual((code, calls), (0, [[str(repo), "branch", "cleanup", "--local", "--json"]]))
|
||||||
|
self.assertEqual(plan, {"repos": [{"repo": str(repo), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}]})
|
||||||
|
cases = (
|
||||||
|
("nothing changed", {}, []),
|
||||||
|
("a worktree index changed", {"worktrees/demo-side/index": "index, refreshed"}, []),
|
||||||
|
("a branch moved", {"refs/heads/side": "b" * 40 + "\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
|
||||||
|
("an upstream or deleteMerged changed", {"config": "[core]\n[branch \"side\"]\n\tdeleteMerged = false\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
|
||||||
|
("origin moved", {"refs/remotes/origin/main": "c" * 40 + "\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
|
||||||
|
("a reftable table list changed", {"reftable/tables.list": "0x000000000001-0x000000000002-a.ref\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
|
||||||
|
)
|
||||||
|
for name, changes, expected in cases:
|
||||||
|
with self.subTest(name=name):
|
||||||
|
code, calls, again = run(changes)
|
||||||
|
self.assertEqual((code, calls, again), (0, expected, plan))
|
||||||
|
(fake.parent / "plan.json").write_text("not json", encoding="utf-8")
|
||||||
|
code, calls, stale = run({"refs/heads/side": "d" * 40 + "\n"})
|
||||||
|
self.assertEqual((code, len(calls), stale["repos"][0]["items"]), (1, 1, items))
|
||||||
|
self.assertRegex(stale["repos"][0]["stale"], r"^replan failed at \d{4}-\d\d-\d\dT")
|
||||||
|
self.assertEqual(run({})[:2], (0, []))
|
||||||
|
(fake.parent / "plan.json").write_text(json.dumps(items), encoding="utf-8")
|
||||||
|
code, calls, plan = run({"refs/heads/side": "e" * 40 + "\n"})
|
||||||
|
self.assertEqual((code, len(calls), plan["repos"][0].get("stale")), (0, 1, None))
|
||||||
|
written.unlink()
|
||||||
|
code, calls, plan = run({})
|
||||||
|
self.assertEqual((code, len(calls), len(plan["repos"])), (0, 1, 1))
|
||||||
|
self.assertEqual(sorted(path.name for path in written.parent.iterdir()), ["branch-tree.json", "local-plan.json"])
|
||||||
|
|
||||||
|
def test_a_sigterm_during_one_slow_replan_keeps_the_plans_written_before_it_and_the_slow_one_is_not_replanned_unchanged(self):
|
||||||
|
fake = self.fake()
|
||||||
|
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "fresh", "local_sha": "a" * 40}]), encoding="utf-8")
|
||||||
|
fast, slow = self.repository("fast"), self.repository("slow")
|
||||||
|
(fake.parent / "slow.txt").write_text(str(slow), encoding="utf-8")
|
||||||
|
self.tree_file.write_text(json.dumps({"repos": [{"path": str(fast), "known": True}, {"path": str(slow), "known": True}]}), encoding="utf-8")
|
||||||
|
self.tree_file.with_name("local-plan.json").write_text(json.dumps({"repos": [{"repo": str(fast), "fingerprint": "old", "items": [{"branch": "old", "local_sha": "b" * 40}]}]}), encoding="utf-8")
|
||||||
|
child = subprocess.Popen(self.argv(fake, 240), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
|
||||||
|
deadline = time.monotonic() + 20
|
||||||
|
while time.monotonic() < deadline and "slow" not in self.written():
|
||||||
|
time.sleep(0.2)
|
||||||
|
os.killpg(child.pid, signal.SIGTERM)
|
||||||
|
child.wait(timeout=20)
|
||||||
|
plans = self.written()
|
||||||
|
self.assertEqual({name: [item["branch"] for item in entry["items"]] for name, entry in plans.items()}, {"fast": ["fresh"], "slow": []})
|
||||||
|
self.assertNotIn("stale", plans["fast"])
|
||||||
|
self.assertRegex(plans["slow"]["stale"], r"^replan started \d{4}-\d\d-\d\dT\S+ has not finished$")
|
||||||
|
self.calls(fake)
|
||||||
|
done = self.plan(fake, dict(os.environ))
|
||||||
|
self.assertEqual((done.returncode, self.calls(fake), self.written()), (0, [], plans))
|
||||||
|
|
||||||
|
def test_a_replan_over_its_budget_is_marked_stale_and_retried_only_after_its_refs_change(self):
|
||||||
|
fake = self.fake()
|
||||||
|
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "fresh", "local_sha": "a" * 40}]), encoding="utf-8")
|
||||||
|
slow = self.repository("slow")
|
||||||
|
self.tree_file.write_text(json.dumps({"repos": [{"path": str(slow), "known": True}]}), encoding="utf-8")
|
||||||
|
self.assertEqual(self.plan(fake, dict(os.environ)).returncode, 0)
|
||||||
|
self.assertEqual(len(self.calls(fake)), 1)
|
||||||
|
(fake.parent / "slow.txt").write_text(str(slow), encoding="utf-8")
|
||||||
|
(slow / ".git" / "refs" / "heads" / "side").write_text("b" * 40 + "\n", encoding="utf-8")
|
||||||
|
started = time.monotonic()
|
||||||
|
timed_out = subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False)
|
||||||
|
self.assertLess(time.monotonic() - started, 20)
|
||||||
|
self.assertEqual((timed_out.returncode, len(self.calls(fake))), (1, 1))
|
||||||
|
stale = self.written()["slow"]
|
||||||
|
self.assertEqual([item["branch"] for item in stale["items"]], ["fresh"])
|
||||||
|
self.assertRegex(stale["stale"], r"^replan timed out at \d{4}-\d\d-\d\dT")
|
||||||
|
again = subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False)
|
||||||
|
self.assertEqual((again.returncode, self.calls(fake), self.written()["slow"]), (0, [], stale))
|
||||||
|
(fake.parent / "slow.txt").unlink()
|
||||||
|
(slow / ".git" / "refs" / "heads" / "side").write_text("c" * 40 + "\n", encoding="utf-8")
|
||||||
|
self.assertEqual(subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False).returncode, 0)
|
||||||
|
self.assertEqual((len(self.calls(fake)), self.written()["slow"].get("stale")), (1, None))
|
||||||
|
|
||||||
|
def test_the_tree_timer_writes_the_local_dry_run_after_the_tree(self):
|
||||||
|
starts = [line.split("=", 1)[1] for line in TREE_UNIT.read_text(encoding="utf-8").splitlines() if line.startswith("ExecStart=")]
|
||||||
|
self.assertEqual(len(starts), 2)
|
||||||
|
self.assertIn("pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json", starts[0])
|
||||||
|
self.assertEqual(starts[1], "/usr/bin/env python3 %h/.local/share/kvb-orchestrator/client/local_plan.py --tree %h/.local/state/pkgmgr/tree/branch-tree.json --pkgmgr %h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr --budget 240")
|
||||||
|
self.assertIn("TimeoutStartSec=600", TREE_UNIT.read_text(encoding="utf-8").splitlines())
|
||||||
|
|
||||||
|
@unittest.skipUnless(INSTALLED_PKGMGR, "no pkgmgr installed")
|
||||||
|
def test_on_files_the_installed_pkgmgr_wrote_the_hub_offers_exactly_the_branches_whose_local_tip_it_deletes(self):
|
||||||
|
root = Path(os.path.realpath(self.outside)) / "pkgmgr"
|
||||||
|
home = root / "home"
|
||||||
|
root.mkdir()
|
||||||
|
repo, _ = capture_local_plan(root, Path(INSTALLED_PKGMGR))
|
||||||
|
config = home / ".config" / "pkgmgr" / "fixture.yml"
|
||||||
|
config.parent.mkdir(parents=True)
|
||||||
|
config.write_text(json.dumps({"directories": {"repositories": str(root)}, "repositories": [{"provider": "github.com", "account": "operator-example-01", "repository": "demo", "directory": str(repo)}]}), encoding="utf-8")
|
||||||
|
env = {
|
||||||
|
"PATH": os.environ["PATH"], "HOME": str(home), "LANG": "C.UTF-8", **GIT_ENV, "DBUS_SESSION_BUS_ADDRESS": f"unix:path={root / 'no-bus'}",
|
||||||
|
"XDG_CACHE_HOME": str(home / ".cache"), "XDG_STATE_HOME": str(home / ".local" / "state"), "PKGMGR_TREE_FILE": str(self.tree_file),
|
||||||
|
}
|
||||||
|
built = subprocess.run([INSTALLED_PKGMGR, "branch", "tree", "--format", "json", "--output", str(self.tree_file)], cwd=root, env=env, capture_output=True, text=True, timeout=300, check=False)
|
||||||
|
self.assertEqual(built.returncode, 0, built.stderr[-2000:])
|
||||||
|
planned = self.plan(Path(INSTALLED_PKGMGR), env)
|
||||||
|
self.assertEqual(planned.returncode, 0, planned.stderr[-2000:])
|
||||||
|
status, _, body = self.get("/api/tree")
|
||||||
|
self.assertEqual(status, 200, body)
|
||||||
|
branches = {entry["name"]: entry for entry in json.loads(body)["repos"][0]["branches"]}
|
||||||
|
shown = {name: (entry["merged_by"], entry["cleanup"] is not None, entry["kept"]) for name, entry in branches.items() if name != "main"}
|
||||||
|
self.assertEqual(shown, {"side": ("ancestor", True, ""), "topic": ("patch", True, ""), "squashed": ("patch", False, f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}; use branch gc (kvb-git-gc, pkgmgr gc)")})
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Reference in new issue
Block a user