docs(skills): carry the CI-artifact mechanics, not just the rule

The rule already lived in triage and still did not fire. `gh run view --log` refuses while a run is in progress, and that refusal read as "logs unavailable" rather than "use the other endpoint", so the investigation fell back on reasoning and produced a root cause the artifact disproves.

The new ci-artifacts skill carries what was missing: per-job logs via `gh api /repos/<owner>/<repo>/actions/jobs/<id>/logs`, which bundle to pull, reading an access log as a timing witness (499 versus 504, ReqTime versus UpstreamTime), and the grep hygiene that stops a substring match — `h2c` inside `veth2cb3f8c` — from becoming a root cause.

triage gains the rules the artifact step lacked: a derived source (a conversation summary, an earlier message, a subagent report) is not evidence; cluster assignments are re-derived after a context compaction; and every per-job verdict must cite the artifact path and line.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5 committed 2026-08-12 18:53:06 +02:00
1 parent d7e2827511
commit f06979ea7f
2 files changed
+99 -2

No files matched your search

+84
View File
@@ -0,0 +1,84 @@
---
name: ci-artifacts
description: >
Get the real evidence out of a CI run: per-job logs even while the run is
still in progress, and the artifacts that hold the cause rather than the
symptom. Trigger whenever a CI failure has to be explained, and from the
triage skill's artifact step. Portable across projects.
---
A job log tells you what broke. An artifact tells you why. Investigating a CI
failure without downloading its artifacts is guessing with extra steps.
## Getting the log
`gh run view --log` refuses to serve anything while a run is unfinished:
run <id> is still in progress; logs will be available when it is complete
That is not a dead end — individual *jobs* that have already finished are
served by the API:
gh api /repos/<owner>/<repo>/actions/jobs/<JOB_ID>/logs
Enumerate the failing jobs and their ids first:
gh run view <RUN_ID> --repo <owner>/<repo> --json jobs \
--jq '.jobs[] | select(.conclusion=="failure") | "\(.databaseId)\t\(.name)"'
Save every log to a file (`/tmp/ci-logs-<run>/<job>.log`) and grep the file
repeatedly. Re-fetching a log to find the line you just lost costs minutes;
grepping the saved copy costs milliseconds.
## Getting the artifacts
gh api /repos/<owner>/<repo>/actions/runs/<RUN_ID>/artifacts --paginate \
--jq '.artifacts[] | "\(.name)\t\(.size_in_bytes)"'
gh run download <RUN_ID> --repo <owner>/<repo> -n <NAME> -D <DIR>
Choose by content, not by name length:
- **Rescue / diagnostics bundles** carry the reverse-proxy access and error log,
the per-service and per-container logs, the journal, and the event stream.
This is where causes live. Prefer them.
- **Test-report bundles** (Playwright and friends) are mostly video and traces
and can run to hundreds of megabytes for one assertion string. Pull one only
when you need the assertion text or the error-context file.
- **Inventory / runtime dumps** settle "what was actually deployed" questions.
Inside a rescue bundle, read in this order: the reverse-proxy log (it shows
status code, upstream, and time-to-response for the exact failing request), the
application container log, then the journal and event stream for restarts and
OOM kills.
An access-log line is a timing witness. A `499` means the *client* gave up; the
server was still working. A `504` or an `upstream timed out ... while reading
upstream` means your backend never answered. `ReqTime` versus `UpstreamTime`
tells you whether the proxy or the upstream burned the seconds.
## Grep hygiene
Two failure modes turn a search into a false conclusion:
- **Clustered flags silently change the pattern.** Write them separately —
`grep -r -a -i -l -F "needle" dir/`, not `-ariel`. A cluster where one letter
takes an argument consumes your pattern and searches for something else, and
the run still exits successfully.
- **Substring matches lie.** A protocol token like `h2c` matches inside
`veth2cb3f8c`; a short id matches inside a hash. Before concluding anything
from a hit, print its context (`grep -o ".\{30\}needle.\{30\}"`) and look at
it. Before concluding anything from a *miss*, prove the command was
well-formed by searching for a string you know is present.
Binary-ish logs need `-a`, or grep reports "binary file matches" and you see
nothing.
## Rules
- Download before theorising. Do it even when the job log looks conclusive: a
log that explains the symptom rarely explains the cause.
- Quote literally, with the file path and line number. Never paraphrase a log
line you did not read, and never reconstruct one from memory.
- A negative result is a finding only when the search was verified sound.
- "The artifact does not settle this" is a legitimate answer. Say it instead of
filling the gap with a plausible mechanism.
+15 -2
View File
@@ -21,7 +21,10 @@ root cause is proven.
actual failing line, not by job name. Jobs whose messages differ in anything actual failing line, not by job name. Jobs whose messages differ in anything
but the interpolated identifiers (host, app, port, path) belong in separate but the interpolated identifiers (host, app, port, path) belong in separate
clusters. clusters.
3. **Download artifacts per cluster.** For each cluster, fetch every artifact of 3. **Download artifacts per cluster.** Use the `ci-artifacts` skill for the
mechanics — it covers per-job logs while a run is still in progress, which
bundle to pull, and the grep hygiene that keeps a false positive from
becoming a root cause. For each cluster, fetch every artifact of
one representative job: reports, rescue diagnostics, inventories, container one representative job: reports, rescue diagnostics, inventories, container
logs. Pull a second member's artifacts only when the representative's logs. Pull a second member's artifacts only when the representative's
evidence does not explain the whole cluster. The job log usually names an evidence does not explain the whole cluster. The job log usually names an
@@ -51,7 +54,17 @@ root cause is proven.
- Executed evidence beats reasoning: read the actual failing log line and the - Executed evidence beats reasoning: read the actual failing log line and the
artifact, not the job title. artifact, not the job title.
- A derived source is not evidence. A conversation summary, your own earlier
message, a cluster table you built ten minutes ago, or a subagent's report
proves nothing on its own. Only the job log, the artifact, the code at the
run's commit, and command output you just ran count. Quote them literally; if
you cannot quote it, you do not know it.
- Re-derive after a context compaction. A summary carried across a compaction
boundary preserves your conclusions but not the evidence under them, and a
cluster assignment that was wrong stays wrong while reading as settled fact.
Before acting on any pre-compaction attribution, rebuild it from the job logs.
- Classify each failure: a real defect (fix it), a flake or infra hiccup (prove - Classify each failure: a real defect (fix it), a flake or infra hiccup (prove
it is transient before dismissing), or already-fixed-by-a-later-commit (verify it is transient before dismissing), or already-fixed-by-a-later-commit (verify
the fix is present in the run's commit before claiming it). the fix is present in the run's commit before claiming it).
- Report a per-job verdict: root cause, the evidence that proves it, and the fix. - Report a per-job verdict: root cause, the artifact path and line that prove
it, and the fix. A verdict that cites only the job log has not finished step 3.