docs(skills): carry the CI-artifact mechanics, not just the rule
The rule already lived in triage and still did not fire. `gh run view --log` refuses while a run is in progress, and that refusal read as "logs unavailable" rather than "use the other endpoint", so the investigation fell back on reasoning and produced a root cause the artifact disproves. The new ci-artifacts skill carries what was missing: per-job logs via `gh api /repos/<owner>/<repo>/actions/jobs/<id>/logs`, which bundle to pull, reading an access log as a timing witness (499 versus 504, ReqTime versus UpstreamTime), and the grep hygiene that stops a substring match — `h2c` inside `veth2cb3f8c` — from becoming a root cause. triage gains the rules the artifact step lacked: a derived source (a conversation summary, an earlier message, a subagent report) is not evidence; cluster assignments are re-derived after a context compaction; and every per-job verdict must cite the artifact path and line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d7e2827511
commit
f06979ea7f
2 files changed
+99
-2
No files matched your search
@@ -0,0 +1,84 @@
|
||||
---
|
||||
name: ci-artifacts
|
||||
description: >
|
||||
Get the real evidence out of a CI run: per-job logs even while the run is
|
||||
still in progress, and the artifacts that hold the cause rather than the
|
||||
symptom. Trigger whenever a CI failure has to be explained, and from the
|
||||
triage skill's artifact step. Portable across projects.
|
||||
---
|
||||
|
||||
A job log tells you what broke. An artifact tells you why. Investigating a CI
|
||||
failure without downloading its artifacts is guessing with extra steps.
|
||||
|
||||
## Getting the log
|
||||
|
||||
`gh run view --log` refuses to serve anything while a run is unfinished:
|
||||
|
||||
run <id> is still in progress; logs will be available when it is complete
|
||||
|
||||
That is not a dead end — individual *jobs* that have already finished are
|
||||
served by the API:
|
||||
|
||||
gh api /repos/<owner>/<repo>/actions/jobs/<JOB_ID>/logs
|
||||
|
||||
Enumerate the failing jobs and their ids first:
|
||||
|
||||
gh run view <RUN_ID> --repo <owner>/<repo> --json jobs \
|
||||
--jq '.jobs[] | select(.conclusion=="failure") | "\(.databaseId)\t\(.name)"'
|
||||
|
||||
Save every log to a file (`/tmp/ci-logs-<run>/<job>.log`) and grep the file
|
||||
repeatedly. Re-fetching a log to find the line you just lost costs minutes;
|
||||
grepping the saved copy costs milliseconds.
|
||||
|
||||
## Getting the artifacts
|
||||
|
||||
gh api /repos/<owner>/<repo>/actions/runs/<RUN_ID>/artifacts --paginate \
|
||||
--jq '.artifacts[] | "\(.name)\t\(.size_in_bytes)"'
|
||||
gh run download <RUN_ID> --repo <owner>/<repo> -n <NAME> -D <DIR>
|
||||
|
||||
Choose by content, not by name length:
|
||||
|
||||
- **Rescue / diagnostics bundles** carry the reverse-proxy access and error log,
|
||||
the per-service and per-container logs, the journal, and the event stream.
|
||||
This is where causes live. Prefer them.
|
||||
- **Test-report bundles** (Playwright and friends) are mostly video and traces
|
||||
and can run to hundreds of megabytes for one assertion string. Pull one only
|
||||
when you need the assertion text or the error-context file.
|
||||
- **Inventory / runtime dumps** settle "what was actually deployed" questions.
|
||||
|
||||
Inside a rescue bundle, read in this order: the reverse-proxy log (it shows
|
||||
status code, upstream, and time-to-response for the exact failing request), the
|
||||
application container log, then the journal and event stream for restarts and
|
||||
OOM kills.
|
||||
|
||||
An access-log line is a timing witness. A `499` means the *client* gave up; the
|
||||
server was still working. A `504` or an `upstream timed out ... while reading
|
||||
upstream` means your backend never answered. `ReqTime` versus `UpstreamTime`
|
||||
tells you whether the proxy or the upstream burned the seconds.
|
||||
|
||||
## Grep hygiene
|
||||
|
||||
Two failure modes turn a search into a false conclusion:
|
||||
|
||||
- **Clustered flags silently change the pattern.** Write them separately —
|
||||
`grep -r -a -i -l -F "needle" dir/`, not `-ariel`. A cluster where one letter
|
||||
takes an argument consumes your pattern and searches for something else, and
|
||||
the run still exits successfully.
|
||||
- **Substring matches lie.** A protocol token like `h2c` matches inside
|
||||
`veth2cb3f8c`; a short id matches inside a hash. Before concluding anything
|
||||
from a hit, print its context (`grep -o ".\{30\}needle.\{30\}"`) and look at
|
||||
it. Before concluding anything from a *miss*, prove the command was
|
||||
well-formed by searching for a string you know is present.
|
||||
|
||||
Binary-ish logs need `-a`, or grep reports "binary file matches" and you see
|
||||
nothing.
|
||||
|
||||
## Rules
|
||||
|
||||
- Download before theorising. Do it even when the job log looks conclusive: a
|
||||
log that explains the symptom rarely explains the cause.
|
||||
- Quote literally, with the file path and line number. Never paraphrase a log
|
||||
line you did not read, and never reconstruct one from memory.
|
||||
- A negative result is a finding only when the search was verified sound.
|
||||
- "The artifact does not settle this" is a legitimate answer. Say it instead of
|
||||
filling the gap with a plausible mechanism.
|
||||
+15
-2
@@ -21,7 +21,10 @@ root cause is proven.
|
||||
actual failing line, not by job name. Jobs whose messages differ in anything
|
||||
but the interpolated identifiers (host, app, port, path) belong in separate
|
||||
clusters.
|
||||
3. **Download artifacts per cluster.** For each cluster, fetch every artifact of
|
||||
3. **Download artifacts per cluster.** Use the `ci-artifacts` skill for the
|
||||
mechanics — it covers per-job logs while a run is still in progress, which
|
||||
bundle to pull, and the grep hygiene that keeps a false positive from
|
||||
becoming a root cause. For each cluster, fetch every artifact of
|
||||
one representative job: reports, rescue diagnostics, inventories, container
|
||||
logs. Pull a second member's artifacts only when the representative's
|
||||
evidence does not explain the whole cluster. The job log usually names an
|
||||
@@ -51,7 +54,17 @@ root cause is proven.
|
||||
|
||||
- Executed evidence beats reasoning: read the actual failing log line and the
|
||||
artifact, not the job title.
|
||||
- A derived source is not evidence. A conversation summary, your own earlier
|
||||
message, a cluster table you built ten minutes ago, or a subagent's report
|
||||
proves nothing on its own. Only the job log, the artifact, the code at the
|
||||
run's commit, and command output you just ran count. Quote them literally; if
|
||||
you cannot quote it, you do not know it.
|
||||
- Re-derive after a context compaction. A summary carried across a compaction
|
||||
boundary preserves your conclusions but not the evidence under them, and a
|
||||
cluster assignment that was wrong stays wrong while reading as settled fact.
|
||||
Before acting on any pre-compaction attribution, rebuild it from the job logs.
|
||||
- Classify each failure: a real defect (fix it), a flake or infra hiccup (prove
|
||||
it is transient before dismissing), or already-fixed-by-a-later-commit (verify
|
||||
the fix is present in the run's commit before claiming it).
|
||||
- Report a per-job verdict: root cause, the evidence that proves it, and the fix.
|
||||
- Report a per-job verdict: root cause, the artifact path and line that prove
|
||||
it, and the fix. A verdict that cites only the job log has not finished step 3.
|
||||
Reference in new issue
Block a user