Skip to content

fix(coverage): report coverage honestly and stop scanned repos steering the engines - #12

Closed
Synvoya wants to merge 3 commits into
agent/engine-bumps-2026-10from
agent/coverage-honesty
Closed

Synvoya wants to merge 3 commits into
agent/engine-bumps-2026-10from
agent/coverage-honesty

Conversation

@Synvoya

@Synvoya Synvoya commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

CodeInspectus no longer reports coverage it did not achieve, and a scanned repository (or the host environment) can no longer make an engine scan less while coverage reads complete.

  • Declared coverage gaps: languages and inputs no bundled rule or engine reads (e.g. php-sast, vue-sast, deno-lock, no-lockfile-npm, wordpress-plugins, actions-unchecked, dev-deps-unscanned, sast-parse-errors) are named. Coverage is partial and CI exits 2 unless the user accepts specific ids (--accept-gaps, MCP accept_gaps, SDK acceptGaps). Accepted ids are recorded in the export.
  • Engines can't be steered: repo .semgrepignore, .trivyignore, trivy.yaml, inherited SEMGREP_*/TRIVY_*/GITLEAKS_* variables and parent-folder names no longer narrow a scan; Opengrep runs under UTF-8 and --no-git-ignore.
  • Errors surface: Opengrep timeouts/internal errors, Trivy inline ignore comments and Gitleaks ignore entries that can hide findings lower coverage instead of being silent.
  • Data fixes: development dependencies are analysed; Trivy findings name package and fixed version; three bundled Opengrep rules no longer hit internal matching errors.
  • Native JS/TS lexer: hardened; its remaining limits are documented in docs/CI-POLICY.md → "Native pack limits" (known-limitation tests included).

Existing finding ids and fingerprints are unchanged. CI impact: repositories with undeclared gaps now exit 2 — see CHANGELOG "Upgrading / CI impact".

Verification

  • Build, 1,827 tests (+12 expected-fail known limitations) and 79 evals pass on this exact tree.
  • 20 public field repositories: no finding lost or changed; all 12 validation repos pass CI with their gaps accepted.

Stacked on #11 (base agent/engine-bumps-2026-10).

…ng the engines

Coverage never reads complete for something CodeInspectus did not analyse:
- Languages and inputs no bundled rule or engine reads are named as stable
  gap ids (php-sast, vue-sast, deno-lock, no-lockfile-npm, wordpress-plugins,
  actions-unchecked, dev-deps-unscanned, sast-parse-errors, ...). Coverage is
  partial and CI exits 2 unless the user accepts specific ids with
  --accept-gaps / accept_gaps / acceptGaps; accepted ids are recorded in the
  export and in history comparisons only where that stays sound.
- Opengrep timeouts, internal errors and parse failures, Trivy inline ignore
  comments and Gitleaks ignore entries that can hide findings now lower
  coverage instead of being silent.

A scanned repository or the host environment can no longer narrow a scan:
- Opengrep ignores the repo's .semgrepignore (managed default ignore list),
  runs under a UTF-8 locale and with --no-git-ignore; Trivy and Gitleaks run
  from an empty managed directory with explicit empty configs; inherited
  SEMGREP_*, OPENGREP_*, TRIVY_* and GITLEAKS_* variables are stripped.

Also:
- Development dependencies are analysed (npm, Yarn, pnpm, Bun, Composer);
  Maven scopes Trivy drops are named as a gap.
- Trivy findings name the package and the fixed version.
- Three bundled Opengrep rules no longer raise internal matching errors.
- Native JS/TS lexer hardening; its remaining limits are documented in
  docs/CI-POLICY.md ("Native pack limits").

Existing finding ids and fingerprints are unchanged. See CHANGELOG
"Upgrading / CI impact".

Signed-off-by: Synvoya <support@synvoya.com>
- Opengrep runs under en_US.UTF-8 on macOS (the runner ships no C.UTF-8)
  and C.UTF-8 elsewhere; a missing locale still fails closed.
- The managed Opengrep ignore list is written with LF line endings even
  from a CRLF (Windows) checkout.
- Tests that assumed POSIX (chmod-based read denial, symlink privileges,
  path separators, the managed Gitleaks link that Windows never uses) now
  assert the correct behaviour per platform.

Signed-off-by: Synvoya <support@synvoya.com>
…list disclosure on Windows

Signed-off-by: Synvoya <support@synvoya.com>
@Synvoya

Synvoya commented Nov 5, 2026

Copy link
Copy Markdown
Owner Author

Folded into #14, the single CodeInspectus 4.0 PR (this change is included there).

@Synvoya Synvoya closed this Nov 5, 2026
Synvoya added a commit that referenced this pull request Nov 5, 2026
osv-scanner 2.7.0 as a fourth engine, the Synvoya gap layer, Synvoya PHP and Node.js Opengrep packs, native JS/TS parsing, quieter output and honest coverage. Folds in #11, #12 and #13. See CHANGELOG.md [4.0.0].

Signed-off-by: Synvoya <support@synvoya.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant