Repository navigation
BUG: acceptance tests still call the removed check and dashboard commands #5068
Description
Description
The steampipe check and steampipe dashboard commands were removed from this repo (cmd/check.go deleted in ce87e26, 2024-05-29), but the acceptance tests still carry code and data written for them. Found while fixing #5064. Line numbers are on develop at 29c9519.
1. A test that passes because the command it runs does not exist
tests/acceptance/test_files/plugin.bats:364-399, verify that steampipe check should bypass plugin requirement detection if installed plugin is local, runs steampipe check all (line 386) and passes when a plugin-requirement warning is absent from the output. The command fails as unknown, so the warning is always absent. The test also clones steampipe-mod-net-insights from GitHub on every run.
2. Skipped tests that call the removed command
tests/acceptance/test_files/cache.bats:140 and :164 call steampipe check benchmark.config_parsing_benchmark. Both tests are skipped with TODO - test using steampipe query command.
3. Branches that no test data reaches
tests/acceptance/test_files/config_precedence.bats:36-39 builds steampipe check all and steampipe dashboard commands. All 19 entries in test_data/source_files/config_tests/workspace_tests.json have "cmd": "query".
4. Exports in tests/acceptance/run.sh that no test reads
Lines 25 and 28-32: SIMPLE_MOD_DIR, CHECK_ALL_MOD, FUNCTIONALITY_TEST_MOD, CONTROL_RENDERING_TEST_MOD, BLANK_DIMENSION_VALUE_TEST_MOD, STRING_LIST_TEST_MOD. CONFIG_PARSING_TEST_MOD (line 26) is read only by the two skipped cache tests above.
5. Mod directories under tests/acceptance/test_data/mods
53 entries. The test files refer by path to three of them: sample_workspace, csv_plugin_test and functionality_test_mod (the last only from snapshot.bats, which CI does not run). Not checked: which of the rest are reached some other way, for example through WORKSPACE_DIR or BAD_TEST_MOD_DIR.
Severity
LOW - Test-only. Item 1 reports a pass for behaviour nothing checks.
Expected Behavior
No acceptance test depends on a command steampipe does not have, and run.sh and test_data/mods hold only what a test uses.
Suggested Fix
- Remove the
plugin.batstest, or rewrite it againststeampipe queryif the plugin-requirement warning can still occur. - Rewrite or remove the two skipped
cache.batstests. - Remove the
checkanddashboardbranches inconfig_precedence.bats. - Remove the unused exports from
run.sh. - Delete the mod directories nothing reaches, after confirming each one.
Related Code
tests/acceptance/test_files/plugin.bats:364-399tests/acceptance/test_files/cache.bats:133-180tests/acceptance/test_files/config_precedence.bats:30-39tests/acceptance/run.sh:22-32tests/acceptance/test_data/mods/
Activity