Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Synvoya dataset digests are over raw bytes; never convert line endings.
detection-db/synvoya/** -text
fixtures/synvoya/dataset/** -text
45 changes: 31 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
#
# Rationale (CG-71 Claim 5): the repo had NO automatic PR/push CI — only the
# manual pin-engines.yml. This adds it, in two tiers that match the tool's own
# engine-gated design (evals E16-E18 self-skip when their security engines are absent):
# engine-gated design (evals E16-E18 and E59-E61 self-skip when their security engines
# or the OSV database they need are absent):
#
# always-run (no engines) — MUST stay green on every PR regardless of engines.
# Runs on Node 24 LTS. A separate Node 22 job proves the declared minimum runtime.
Expand All @@ -12,10 +13,13 @@
# Redaction is covered here by eval E03; the gitleaks-backed redaction-e2e is
# engine-dependent and runs in the engine job below.
#
# engine (linux/amd64) — fetches + cosign/checksum-verifies + SHA-pins the three
# engine binaries and pulls the Trivy vuln DB, then runs the FULL eval suite with
# engine-backed, native-pack, parity, rescan, provenance, and redaction gates, plus the gitleaks-backed redaction-e2e and a real
# fixture scan. Proven feasible in CG-77 recon: every engine has a non-null
# engine (linux/amd64) — fetches + cosign/checksum-verifies + SHA-pins the engine
# binaries and pulls the Trivy vuln DB and the OSV PyPI, npm, Packagist, Go and GitHub
# Actions databases (E87-E92 need npm, Go and GitHub Actions; E61 needs the crates.io
# database absent, so crates.io is never installed). It then
# runs the FULL eval suite with engine-backed, native-pack, parity, rescan,
# provenance, and redaction gates, plus the gitleaks-backed redaction-e2e and a
# real fixture scan. Proven feasible in CG-77 recon: every engine has a non-null
# linux-x64 pin and a fresh verified install reproduces the committed SHAs.
#
# runtime-portability — repeats the native engine verification, complete eval,
Expand All @@ -30,6 +34,8 @@
# .sigstore.json/checksums; action code
# *.sigstore.dev (fulcio/rekor/tuf-repo-cdn) cosign keyless verify-blob (tlog + TUF root)
# mirror.gcr.io, ghcr.io Trivy vuln-DB OCI pull (install-time only)
# osv-vulnerabilities.storage.googleapis.com OSV database zips (install-time only;
# scans themselves run fully offline)
# registry.npmjs.org npm ci
#
# Third-party actions are pinned to a commit SHA (mirrors the project's engine
Expand All @@ -48,10 +54,11 @@ concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

# CI runs only in the public repo (free Actions); the private dev repo never runs it (Hibi, 2026-11-02).
jobs:
dco:
name: contributor sign-off (DCO)
if: github.event_name == 'pull_request'
if: github.repository == 'Synvoya/codeinspectus' && github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
Expand All @@ -76,6 +83,7 @@ jobs:

always:
name: always-run (no engines)
if: github.repository == 'Synvoya/codeinspectus'
runs-on: ubuntu-latest
steps:
# audit mode — logs egress for a future block-mode allowlist (CG-78); does not restrict.
Expand Down Expand Up @@ -112,6 +120,7 @@ jobs:

node22-compatibility:
name: minimum runtime (Node 22)
if: github.repository == 'Synvoya/codeinspectus'
runs-on: ubuntu-latest
steps:
- uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
Expand All @@ -134,6 +143,7 @@ jobs:

engine:
name: engine (linux/amd64)
if: github.repository == 'Synvoya/codeinspectus'
runs-on: ubuntu-latest
steps:
# audit mode — logs egress for a future block-mode allowlist (CG-78); does not restrict.
Expand All @@ -157,22 +167,26 @@ jobs:
test "$actual" = "linux-x64"
- name: Repair engines — verify immutable shipped pins + install + Trivy DB
run: node dist/index.js repair-engines --refresh-db
- name: Install the OSV databases the evals need (explicit list; crates.io is deliberately not downloaded)
run: node dist/index.js repair-engines osv-scanner --osv-ecosystems "PyPI,npm,Packagist,Go,GitHub Actions"
- name: Assert user repair did not mutate the packaged lockfile
run: git diff --exit-code -- engines.lock.json
- name: Verify installed binaries against the lockfile (native OS)
run: node dist/index.js verify-engines
- name: Eval — full suite; REQUIRE E16/E18 Opengrep + E17 Trivy to RUN (0 skipped)
- name: Eval — full suite; REQUIRE E16/E18 Opengrep + E17 Trivy + E59-E61 osv-scanner to RUN (0 skipped)
shell: bash
run: |
# The engine tier EXISTS to prove Opengrep + Trivy actually execute. A skip
# The engine tier EXISTS to prove Opengrep + Trivy + osv-scanner actually execute. A skip
# here is a FAILURE, not acceptable degradation: if the Trivy DB pull silently
# failed or an engine did not run, E16-E18 self-skip and the eval harness
# exits 0 — a false green. Asserting 0 skipped turns that into a RED instead.
# failed, an OSV database was not installed, or an engine did not run, E16-E18, E59-E61
# and E87-E92 self-skip and the eval harness exits 0 — a false green. Asserting 0 skipped
# turns that into a RED instead. E61 also needs the OSV crates.io database ABSENT (its
# missing-database case), which is why crates.io is never installed.
out="${RUNNER_TEMP:-/tmp}/eval.out"
set -o pipefail
npm run eval | tee "$out"
grep -qE '0 failed, 0 skipped' "$out" || {
echo "FAIL: engine tier requires 0 skipped — E16/E18 Opengrep + E17 Trivy must RUN, not skip (check the Trivy DB pull from mirror.gcr.io / ghcr.io)."
echo "FAIL: engine tier requires 0 skipped — E16/E18 Opengrep + E17 Trivy + E59-E61 osv-scanner must RUN, not skip (check the Trivy DB pull from mirror.gcr.io / ghcr.io and the OSV database downloads)."
exit 1
}
- name: Redaction e2e — gitleaks-detected non-pattern secrets stay redacted
Expand All @@ -182,6 +196,7 @@ jobs:

runtime-portability:
name: runtime (${{ matrix.platform }})
if: github.repository == 'Synvoya/codeinspectus'
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand All @@ -191,7 +206,7 @@ jobs:
platform: win32-x64
- os: macos-15-intel
platform: darwin-x64
- os: macos-14
- os: macos-15
platform: darwin-arm64
- os: ubuntu-24.04-arm
platform: linux-arm64
Expand Down Expand Up @@ -224,18 +239,20 @@ jobs:
test "$actual" = "$EXPECTED_PLATFORM"
- name: Repair engines — verify immutable shipped pins + install + Trivy DB
run: node dist/index.js repair-engines --refresh-db
- name: Install the OSV databases the evals need (explicit list; crates.io is deliberately not downloaded)
run: node dist/index.js repair-engines osv-scanner --osv-ecosystems "PyPI,npm,Packagist,Go,GitHub Actions"
- name: Assert user repair did not mutate the packaged lockfile
run: git diff --exit-code -- engines.lock.json
- name: Verify installed binaries against the lockfile (native OS)
run: node dist/index.js verify-engines
- name: Eval — full suite; REQUIRE E16/E18 Opengrep + E17 Trivy to RUN (0 skipped)
- name: Eval — full suite; REQUIRE E16/E18 Opengrep + E17 Trivy + E59-E61 osv-scanner to RUN (0 skipped)
shell: bash
run: |
out=".codeinspectus-eval.out"
set -o pipefail
npm run eval | tee "$out"
grep -qE '0 failed, 0 skipped' "$out" || {
echo "FAIL: portability tier requires 0 skipped — E16/E18 Opengrep + E17 Trivy must RUN."
echo "FAIL: portability tier requires 0 skipped — E16/E18 Opengrep + E17 Trivy + E59-E61 osv-scanner must RUN (check the OSV database downloads)."
exit 1
}
- name: Redaction e2e — gitleaks-detected non-pattern secrets stay redacted
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pin-engines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: macos-14 # Apple Silicon
- os: macos-15 # Apple Silicon
platform: darwin-arm64
- os: macos-15-intel # Intel
platform: darwin-x64
Expand Down
1 change: 1 addition & 0 deletions .gitleaks.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ paths = [
'''fixtures/vulnerable-app/src/config\.ts''',
'''evals/run-evals\.ts''',
'''fixtures/secret-rls-corpus/''',
'''fixtures/coverage-honesty-corpus/gitleaksignore-relative/config\.yml''',
]
Loading
Loading