Repository navigation
CI: real-tool smoke for trivy, with a digest-pinned vuln DB - #2372
Merged
Merged
Conversation
Adds real semgrep 1.157.0 (repo rules, no registry pack) and njsscan 0.4.2 canaries to the real-hub job. They install from their own hash-locked scripts/real-hub-requirements-semgrep.txt venv because semgrep pins click 8.1.8, jsonschema 4.25.1 and wcmatch 8.5.2, older than the versions the other tools resolve to. The trufflehog canary matches the exact detector name Github. provision_tools.sh pins checkov 3.4.5. review_checks.sh runs njsscan with the sandboxed HOME and no semgrep metrics, asserted by the hub stub test. Tests: 21/21 (hub secrets-node stub test; k8s-js and js-iac-license hub tests pass; real-hub canaries run in CI) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The real-hub semgrep and njsscan venv now resolves semgrep 1.182.1, the version provision_tools.sh installs for users (njsscan 0.4.2 is compatible). Its Linux wheels are manylinux_2_34, so the lock is re-derived for that platform. test-provision-tools.sh now expects the checkov==3.4.5 pin. Tests: 61/61 (test-provision-tools.sh) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The real-hub job now installs tflint 0.66.0 and kics 2.1.20 (sha256 from each release checksums.txt; kics queries from the release commit fetched by full sha and verified), cppcheck 2.13.0-2ubuntu3 (exact version from the signed noble archive; job pinned to ubuntu-24.04) and DevSkim CLI 1.0.100 (nupkg sha512 = nuget.org catalog packageHash). Each gets a fictional canary asserted through review_checks.sh: tool ran, expected rule on the canary file, no not-run or NOT CHECKED line. syft is skipped: the hub never counts it as ran. Tests: kics canary passes locally; tflint, cppcheck and devskim canaries run only in CI (not runnable in the local sandbox). Gates: bash -n, actionlint, privacy, size, size-ratchet, test-provision-tools.sh all green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x comments cppcheck is now the noble .deb fetched from its stable Launchpad URL and checked with sha256 (an exact apt version breaks once superseded); runtime deps still come from apt. The DevSkim nupkg is also checked with `dotnet nuget verify` when the runner's dotnet has that verb. The real-hub job header no longer calls cppcheck stub-only, and the raw kics rerun prints results.json even when kics exits non-zero. Tests: 5/5 gates (bash -n, actionlint, privacy, size, size-ratchet); test-provision-tools.sh passes against the committed provision_tools.sh (the worktree copy is stale and unrestorable in this sandbox). Canaries unchanged: kics ran locally earlier, tflint/cppcheck/devskim run in CI only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extend the real-hub job to run govulncheck 1.9.0 and grype 0.121.3 through review_checks.sh on fictional canaries, asserting each RAN with its expected rule and no not-run or NOT CHECKED line. grype: linux_amd64 archive checked against the release checksums file (and a literal hash). The canary is a one-component CycloneDX SBOM (lodash 4.17.15), because the hub points grype at SBOM files only, never at a lockfile. govulncheck: no release binaries, so built with go install on the pinned Go 1.26.11 (GOTOOLCHAIN=local), module hash asserted from go version -m, source verified by the Go checksum DB. The canary calls yaml.Unmarshal of the real module gopkg.in/yaml.v2 2.2.2; go mod tidy fills go.sum and the module cache, then the hub runs with GOPROXY=off. Both vulnerability DBs are ONLINE and NOT pinned: govulncheck has no env override for -db, so it reads vuln.go.dev at run time; the grype DB comes from grype db update (checksum-verified against its listing; a dated archive pin would rot when upstream rotates it out). Locally both tools passed the hub canary against mirrored DBs; the real CI run is the first with the live endpoints. Tests: 7/7 (bash -n, actionlint, privacy grep, size gate, size-ratchet, fragments, scripts/test-provision-tools.sh 61 ok 0 fail) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d grype smoke Raise the real-hub timeout to 25 minutes. Retry up to 3 times with backoff around the govulncheck go install, grype db update, go mod tidy for the canary, and the hub run while govulncheck or grype reads as not run (vuln.go.dev is fetched at run time). The grype canary now also accepts the CVE aliases of its GHSA advisories. The canary go.mod stays at the repo root: the hub runs govulncheck only from a root go.mod, so moving it would make the tool NOT CHECKED. Instead the test now fails if gosec or staticcheck report any row on the govulncheck canary package; gosec's own canary rows are unchanged in a local run. Tests: 7/7 (bash -n, actionlint, privacy grep, size gate, size-ratchet, fragments, scripts/test-provision-tools.sh 61 ok 0 fail) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts in ci.yml and test-review-checks-real-hub.sh resolved to the smoke-7 side (main carried only the squashed smoke-6 content). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run real trivy 0.76.2 through review_checks.sh in the real-hub job on a Dockerfile misconfig canary (DS-0001/DS-0002) and an npm lockfile canary (lodash 4.17.15). The binary is pinned by sha256 from the release checksums file; the vulnerability DB is pulled by OCI digest with bounded retries; the misconfig checks are the bundle embedded in the pinned binary (TRIVY_SKIP_CHECK_UPDATE=true). The test also fails on a secret-only fallback. codeql is not added: its CLI bundle and per-language database builds are too heavy for a per-push job. Tests: 7/7 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Verify the release checksums file's sigstore bundle with cosign verify-blob (cosign pinned by sha256; identity pinned to the trivy release workflow at the release tag, issuer to GitHub Actions OIDC). After the DB download, assert the log shows the pinned digest reference and a DB was written. The trivy canary now asserts DS-0001 and DS-0002 each, the debug raw command passes --cache-dir, and the redundant checksum count grep is dropped. Tests: 7/7 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Real kubescape appends .json to an --output path that lacks it, so review_checks.sh read an empty report and marked kubescape not run on every real run. The hub now writes and reads a path ending in .json. The k8s stub test mimics the suffix rule, so it fails on the old code. The real-hub CI job now runs kubescape 3.0.48 (sha256 from the release checksums) with the regolibrary v1.0.316 nsa and cis-v1.23-t1.0.1 frameworks (sha256 recorded at pin time; that release publishes none) on a fictional privileged-pod canary, asserting control C-0057. Tests: 6/6 (k8s stub test 23 ok, real kubescape canary through the hub, bash -n, actionlint, changelog fragments, test-provision-tools) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/ci.yml # scripts/test-review-checks-real-hub.sh
# Conflicts: # .github/workflows/ci.yml # scripts/test-review-checks-real-hub.sh
# Conflicts: # scripts/test-review-checks-real-hub.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run real trivy 0.76.2 through review_checks.sh in the real-hub job on a
Dockerfile misconfig canary (DS-0001/DS-0002) and an npm lockfile canary
(lodash 4.17.15). The binary is pinned by sha256 from the release checksums
file; the vulnerability DB is pulled by OCI digest with bounded retries; the
misconfig checks are the bundle embedded in the pinned binary
(TRIVY_SKIP_CHECK_UPDATE=true). The test also fails on a secret-only fallback.
codeql is not added: its CLI bundle and per-language database builds are too
heavy for a per-push job.
Tests: 7/7
🤖 Generated with Claude Code