docs(skills): carry the CI-artifact mechanics, not just the rule
The rule already lived in triage and still did not fire. `gh run view --log` refuses while a run is in progress, and that refusal read as "logs unavailable" rather than "use the other endpoint", so the investigation fell back on reasoning and produced a root cause the artifact disproves. The new ci-artifacts skill carries what was missing: per-job logs via `gh api /repos/<owner>/<repo>/actions/jobs/<id>/logs`, which bundle to pull, reading an access log as a timing witness (499 versus 504, ReqTime versus UpstreamTime), and the grep hygiene that stops a substring match — `h2c` inside `veth2cb3f8c` — from becoming a root cause. triage gains the rules the artifact step lacked: a derived source (a conversation summary, an earlier message, a subagent report) is not evidence; cluster assignments are re-derived after a context compaction; and every per-job verdict must cite the artifact path and line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d7e2827511
commit
f06979ea7f
2 files changed
+99
-2
No files matched your search
@@ -0,0 +1,84 @@
|
|||||||
|
---
|
||||||
|
name: ci-artifacts
|
||||||
|
description: >
|
||||||
|
Get the real evidence out of a CI run: per-job logs even while the run is
|
||||||
|
still in progress, and the artifacts that hold the cause rather than the
|
||||||
|
symptom. Trigger whenever a CI failure has to be explained, and from the
|
||||||
|
triage skill's artifact step. Portable across projects.
|
||||||
|
---
|
||||||
|
|
||||||
|
A job log tells you what broke. An artifact tells you why. Investigating a CI
|
||||||
|
failure without downloading its artifacts is guessing with extra steps.
|
||||||
|
|
||||||
|
## Getting the log
|
||||||
|
|
||||||
|
`gh run view --log` refuses to serve anything while a run is unfinished:
|
||||||
|
|
||||||
|
run <id> is still in progress; logs will be available when it is complete
|
||||||
|
|
||||||
|
That is not a dead end — individual *jobs* that have already finished are
|
||||||
|
served by the API:
|
||||||
|
|
||||||
|
gh api /repos/<owner>/<repo>/actions/jobs/<JOB_ID>/logs
|
||||||
|
|
||||||
|
Enumerate the failing jobs and their ids first:
|
||||||
|
|
||||||
|
gh run view <RUN_ID> --repo <owner>/<repo> --json jobs \
|
||||||
|
--jq '.jobs[] | select(.conclusion=="failure") | "\(.databaseId)\t\(.name)"'
|
||||||
|
|
||||||
|
Save every log to a file (`/tmp/ci-logs-<run>/<job>.log`) and grep the file
|
||||||
|
repeatedly. Re-fetching a log to find the line you just lost costs minutes;
|
||||||
|
grepping the saved copy costs milliseconds.
|
||||||
|
|
||||||
|
## Getting the artifacts
|
||||||
|
|
||||||
|
gh api /repos/<owner>/<repo>/actions/runs/<RUN_ID>/artifacts --paginate \
|
||||||
|
--jq '.artifacts[] | "\(.name)\t\(.size_in_bytes)"'
|
||||||
|
gh run download <RUN_ID> --repo <owner>/<repo> -n <NAME> -D <DIR>
|
||||||
|
|
||||||
|
Choose by content, not by name length:
|
||||||
|
|
||||||
|
- **Rescue / diagnostics bundles** carry the reverse-proxy access and error log,
|
||||||
|
the per-service and per-container logs, the journal, and the event stream.
|
||||||
|
This is where causes live. Prefer them.
|
||||||
|
- **Test-report bundles** (Playwright and friends) are mostly video and traces
|
||||||
|
and can run to hundreds of megabytes for one assertion string. Pull one only
|
||||||
|
when you need the assertion text or the error-context file.
|
||||||
|
- **Inventory / runtime dumps** settle "what was actually deployed" questions.
|
||||||
|
|
||||||
|
Inside a rescue bundle, read in this order: the reverse-proxy log (it shows
|
||||||
|
status code, upstream, and time-to-response for the exact failing request), the
|
||||||
|
application container log, then the journal and event stream for restarts and
|
||||||
|
OOM kills.
|
||||||
|
|
||||||
|
An access-log line is a timing witness. A `499` means the *client* gave up; the
|
||||||
|
server was still working. A `504` or an `upstream timed out ... while reading
|
||||||
|
upstream` means your backend never answered. `ReqTime` versus `UpstreamTime`
|
||||||
|
tells you whether the proxy or the upstream burned the seconds.
|
||||||
|
|
||||||
|
## Grep hygiene
|
||||||
|
|
||||||
|
Two failure modes turn a search into a false conclusion:
|
||||||
|
|
||||||
|
- **Clustered flags silently change the pattern.** Write them separately —
|
||||||
|
`grep -r -a -i -l -F "needle" dir/`, not `-ariel`. A cluster where one letter
|
||||||
|
takes an argument consumes your pattern and searches for something else, and
|
||||||
|
the run still exits successfully.
|
||||||
|
- **Substring matches lie.** A protocol token like `h2c` matches inside
|
||||||
|
`veth2cb3f8c`; a short id matches inside a hash. Before concluding anything
|
||||||
|
from a hit, print its context (`grep -o ".\{30\}needle.\{30\}"`) and look at
|
||||||
|
it. Before concluding anything from a *miss*, prove the command was
|
||||||
|
well-formed by searching for a string you know is present.
|
||||||
|
|
||||||
|
Binary-ish logs need `-a`, or grep reports "binary file matches" and you see
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- Download before theorising. Do it even when the job log looks conclusive: a
|
||||||
|
log that explains the symptom rarely explains the cause.
|
||||||
|
- Quote literally, with the file path and line number. Never paraphrase a log
|
||||||
|
line you did not read, and never reconstruct one from memory.
|
||||||
|
- A negative result is a finding only when the search was verified sound.
|
||||||
|
- "The artifact does not settle this" is a legitimate answer. Say it instead of
|
||||||
|
filling the gap with a plausible mechanism.
|
||||||
+15
-2
@@ -21,7 +21,10 @@ root cause is proven.
|
|||||||
actual failing line, not by job name. Jobs whose messages differ in anything
|
actual failing line, not by job name. Jobs whose messages differ in anything
|
||||||
but the interpolated identifiers (host, app, port, path) belong in separate
|
but the interpolated identifiers (host, app, port, path) belong in separate
|
||||||
clusters.
|
clusters.
|
||||||
3. **Download artifacts per cluster.** For each cluster, fetch every artifact of
|
3. **Download artifacts per cluster.** Use the `ci-artifacts` skill for the
|
||||||
|
mechanics — it covers per-job logs while a run is still in progress, which
|
||||||
|
bundle to pull, and the grep hygiene that keeps a false positive from
|
||||||
|
becoming a root cause. For each cluster, fetch every artifact of
|
||||||
one representative job: reports, rescue diagnostics, inventories, container
|
one representative job: reports, rescue diagnostics, inventories, container
|
||||||
logs. Pull a second member's artifacts only when the representative's
|
logs. Pull a second member's artifacts only when the representative's
|
||||||
evidence does not explain the whole cluster. The job log usually names an
|
evidence does not explain the whole cluster. The job log usually names an
|
||||||
@@ -51,7 +54,17 @@ root cause is proven.
|
|||||||
|
|
||||||
- Executed evidence beats reasoning: read the actual failing log line and the
|
- Executed evidence beats reasoning: read the actual failing log line and the
|
||||||
artifact, not the job title.
|
artifact, not the job title.
|
||||||
|
- A derived source is not evidence. A conversation summary, your own earlier
|
||||||
|
message, a cluster table you built ten minutes ago, or a subagent's report
|
||||||
|
proves nothing on its own. Only the job log, the artifact, the code at the
|
||||||
|
run's commit, and command output you just ran count. Quote them literally; if
|
||||||
|
you cannot quote it, you do not know it.
|
||||||
|
- Re-derive after a context compaction. A summary carried across a compaction
|
||||||
|
boundary preserves your conclusions but not the evidence under them, and a
|
||||||
|
cluster assignment that was wrong stays wrong while reading as settled fact.
|
||||||
|
Before acting on any pre-compaction attribution, rebuild it from the job logs.
|
||||||
- Classify each failure: a real defect (fix it), a flake or infra hiccup (prove
|
- Classify each failure: a real defect (fix it), a flake or infra hiccup (prove
|
||||||
it is transient before dismissing), or already-fixed-by-a-later-commit (verify
|
it is transient before dismissing), or already-fixed-by-a-later-commit (verify
|
||||||
the fix is present in the run's commit before claiming it).
|
the fix is present in the run's commit before claiming it).
|
||||||
- Report a per-job verdict: root cause, the evidence that proves it, and the fix.
|
- Report a per-job verdict: root cause, the artifact path and line that prove
|
||||||
|
it, and the fix. A verdict that cites only the job log has not finished step 3.
|
||||||
Reference in new issue
Block a user