[jiefenghuang]

Problem

The scheduled dump_load (tidb) job timed out in Test Load & Dump with Json Fast Mode (three runs). It hung on the very first line of prepare_test:

19:12:55  + [[ ! -f /jfs/.config ]]
19:42:50  ##[error]The action ... has timed out after 30 minutes.

while /var/log/juicefs.log filled with a restart storm:

juicefs[15812] <FATAL>: database is not formatted, please run `juicefs format ...` first
juicefs[6706]  <ERROR>: mount process 15812: exit status 1, will restart in 1 second

Root cause

Three defects compose (verified against run 32660109543, job 97244858931):

1. kill_tiup_playground SIGKILLed the running mount. It reaped ports with lsof -ti tcp:$port, but lsof -i tcp:PORT matches clients of that port too, not just listeners. A JuiceFS mount holds MySQL connections to TiDB on 4000, so it went down with the server:

++ lsof -ti tcp:4000
+ pids='3958
6717'
+ kill -9 3958 6717

3958 is the old tidb-server; 6717 is the /jfs mount worker (child of supervisor 6706, mounted at 19:12:08).

2. install_tidb had no reuse guard. install_tikv returns early when the cluster is already healthy; install_tidb did not, so it rebuilt the playground on every step and tiup clean --all wiped the metadata of the still-mounted volume — hence database is not formatted. This is why only the tidb matrix entry failed.

3. The hang itself. The supervisor holds a dup of the /dev/fuse fd, so when the worker is killed the FUSE connection is never aborted: /jfs neither returns ENOTCONN nor unmounts, and every syscall blocks. umount_jfs began with [[ ! -f "$mp/.config" ]] — a stat() served by the dead daemon — so the job wedged there with no timeout.

Contributing: dump_load.sh ran start_meta_engine before any cleanup, and its EXIT trap only echoed the seed, so the Binary step routinely left /jfs mounted across the engine swap.

Changes

• kill_tiup_playground filters to listening sockets and verifies ownership before killing (/proc/<pid>/exe under the tiup home, falling back to the process name). Spared listeners are reported. Genuine orphaned pd/tikv/tidb servers are still reaped, so the original tiup#2057 workaround keeps working.
• install_tidb reuses a healthy playground, mirroring install_tikv.
• umount_jfs decides from the mount table instead of reading through the mount, bounds juicefs status, and falls back to the process table when the given meta URL doesn't know the mount — /jfs2 is used by both $META_URL2 and sqlite3://test2.db, so the session lookup can legitimately miss and the pid must still be waited on.
• dump_load.sh drops stale mounts before start_meta_engine replaces the engine (the old engine is still up there, so sessions resolve and processes are reaped cleanly) and unmounts on exit.

timeout is GNU coreutils and absent from a stock macOS runner (test-mac/mac_commands.sh sources common.sh), so run_with_timeout falls back to a watchdog. Notably not perl's alarm+exec: the Go runtime installs its own SIGALRM handler and silently ignores the signal, so it would never interrupt ./juicefs — measured 3s timeout vs. 20s actual runtime, exit 0.

Testing

CI-only shell changes, so no Go test target applies. Validated with a harness covering 22 cases, all passing:

• The bug reproduces and is fixed: unfiltered lsof returns both server and client pids; with -sTCP:LISTEN the client is excluded and survives kill_tiup_playground.
• A real playground component listening on the same port is still killed (guards against over-correcting).
• is_tiup_component accepts both branches for genuine components and rejects juicefs/python3/mysqld/dockerd.
• is_mounted against the real mount table, including /jfs vs /jfs2 prefix collisions.
• run_with_timeout aborts a Go binary in the no-coreutils path and passes through stdout/exit status.
• The process-table fallback finds the owning pid, matches the mount point exactly, and does not mistake juicefs umount <mp> for the daemon.

bash -n passes on all three files; shellcheck -S error reports no new findings.

Out of scope

The client-side defect remains: launchMount's restart loop is effectively unbounded (its give-up guard only covers the first 10s after initial launch), and because the supervisor pins the FUSE fd, a mount whose metadata engine is destroyed hangs indefinitely instead of failing fast with ENOTCONN. That affects any such scenario, not just CI, and is better addressed separately with its own regression test.

Fixes #7458