Skip to content

BUG: acceptance tests still call the removed check and dashboard commands #5068

Closed

Description

@kaidaguerre

Description

The steampipe check and steampipe dashboard commands were removed from this repo (cmd/check.go deleted in ce87e26, 2024-05-29), but the acceptance tests still carry code and data written for them. Found while fixing #5064. Line numbers are on develop at 29c9519.

1. A test that passes because the command it runs does not exist

tests/acceptance/test_files/plugin.bats:364-399, verify that steampipe check should bypass plugin requirement detection if installed plugin is local, runs steampipe check all (line 386) and passes when a plugin-requirement warning is absent from the output. The command fails as unknown, so the warning is always absent. The test also clones steampipe-mod-net-insights from GitHub on every run.

2. Skipped tests that call the removed command

tests/acceptance/test_files/cache.bats:140 and :164 call steampipe check benchmark.config_parsing_benchmark. Both tests are skipped with TODO - test using steampipe query command.

3. Branches that no test data reaches

tests/acceptance/test_files/config_precedence.bats:36-39 builds steampipe check all and steampipe dashboard commands. All 19 entries in test_data/source_files/config_tests/workspace_tests.json have "cmd": "query".

4. Exports in tests/acceptance/run.sh that no test reads

Lines 25 and 28-32: SIMPLE_MOD_DIR, CHECK_ALL_MOD, FUNCTIONALITY_TEST_MOD, CONTROL_RENDERING_TEST_MOD, BLANK_DIMENSION_VALUE_TEST_MOD, STRING_LIST_TEST_MOD. CONFIG_PARSING_TEST_MOD (line 26) is read only by the two skipped cache tests above.

5. Mod directories under tests/acceptance/test_data/mods

53 entries. The test files refer by path to three of them: sample_workspace, csv_plugin_test and functionality_test_mod (the last only from snapshot.bats, which CI does not run). Not checked: which of the rest are reached some other way, for example through WORKSPACE_DIR or BAD_TEST_MOD_DIR.

Severity

LOW - Test-only. Item 1 reports a pass for behaviour nothing checks.

Expected Behavior

No acceptance test depends on a command steampipe does not have, and run.sh and test_data/mods hold only what a test uses.

Suggested Fix

  1. Remove the plugin.bats test, or rewrite it against steampipe query if the plugin-requirement warning can still occur.
  2. Rewrite or remove the two skipped cache.bats tests.
  3. Remove the check and dashboard branches in config_precedence.bats.
  4. Remove the unused exports from run.sh.
  5. Delete the mod directories nothing reaches, after confirming each one.

Related Code

  • tests/acceptance/test_files/plugin.bats:364-399
  • tests/acceptance/test_files/cache.bats:133-180
  • tests/acceptance/test_files/config_precedence.bats:30-39
  • tests/acceptance/run.sh:22-32
  • tests/acceptance/test_data/mods/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions