[mikehardy]

Stack position: 3 of 4. Base: generic-tooling-fixes. Merge after #9447. Stack order: #9446 (build concurrency cap) → #9447 (tooling and test fixes) → this PR → #9448 (shared-host e2e hardening).

Draft: the local serial Android, iOS and macOS e2e runs passed (see Test Plan). Waiting on review before this is marked ready.

Description

An e2e run should only ever touch what it started. Until now, yarn tests:e2e:release, the Metro start, the Jet harness and yarn test:full freed resources by identity: every listener on the slot's ports, killall by app name, the simulator or AVD by name, and any stray emulator-* serial. When two checkouts shared an identity or a port, one could kill the other's live run.

This PR records what each run starts and frees only that. Wiping the whole machine becomes a manual step.

Run record

• New run record in scripts/e2e/lib/e2e-run-record.sh and tests/e2e/runRecord.js, stored under .e2e-run/<slot-N|serial>/ (git-ignored; RNFB_E2E_RUN_RECORD_DIR overrides the location).
• Recorded at spawn, so an interrupted start is still freed: Metro, the Firebase emulator suite and Jet (the harness records the Jet it spawns on Android and iOS; on macOS, Jet records its own process from tests-macos/.jetrc.js), plus the macOS app and the emulator or simulator instance the run booted or used.
• Pid entries carry the process start time, so a reused pid never counts as owned. A process is owned when it or an ancestor is recorded.
• Device entries carry an instance token (qemu pid + start time with the serial from qemu -port, or the launchd_sim pid + start time). Devices are re-recorded after every harness reboot, including when the reboot throws, and when Detox booted one but the test never started. Another owner's emulator, on any port, is never recorded.
• The harness cold boot stops only this run's emulator (emu kill only after emu avd name matches; one SIGKILL if SIGTERM is ignored). It no longer runs pkill by AVD name.

Release and check

• yarn tests:e2e:release frees recorded resources only: recorded listeners on the identity's ports, the recorded macOS app pid (never killall), and with --devices the recorded simulator by UDID and the recorded emulator (or one a recorded process spawned), only while that exact instance is alive. The adb server is never stopped. Anything else is reported owner=… left running with its command, working directory and a hint, and release exits non-zero if the resource is still busy.
• yarn tests:e2e:check labels every holder owner=this-checkout (release frees it), owner=this-checkout-unrecorded (prints yarn tests:e2e:release --include-unrecorded --only <category>) or owner=unrecorded (names the first free slot and its export-slot-env override).
• Stray emulator-* serials are BUSY only when this checkout recorded or spawned them; others are INFO with owner, working directory and AVD. Another owner's emulator on the identity's own serial is BUSY. Serial iOS check fails only when the identity's own simulator is booted.

Fail fast instead of taking over

• Metro start, the Jet harness (killTcpPortListener) and macOS .jetrc.js before() fail fast on a holder they did not start.
• After a failed emulator-suite start, cleanup stops only the suite's own process tree and owned listeners.
• Before the outer Jet retry, the drain stops the Jet the failed attempt is running now. The attempt context keeps the Jet session, whose process follows launch-retry respawns. The drain first abandons the session, so the attempt's still-running orchestrate cannot respawn Jet, then kills the current Jet's process tree while its root is alive. Before, it stopped only the attempt's first (already respawned) Jet and relied on lsof to find the rest; when lsof listed nothing, the wait for the Jet port to close timed out. A close-wait timeout now names what lsof sees on the port and each holder's owner.
• The drain's time is bounded:
  • It starts one app terminate and waits for it at most 15 s. Detox's terminateApp has no timeout of its own; on iOS one ran 81 s before the orchestrate wait could start.
  • It then waits at most 30 s for the failed attempt's orchestrate, then up to 30 s for the Jet port to close (Android: the host clear, which first force-stops the two test apps, up to 15 s each, and waits about 30 s for the instrumentation to stop, slightly longer when its last adb poll is slow). On Android the drain also force-stops the two apps once before the orchestrate wait. If the instrumentation wait or the port close times out, it throws and the drain fails instead of waiting longer.
  • Last, it joins the same terminate until 90 s after it started. A fast terminate adds no time, and a slow one cannot land on the next attempt's freshly launched app. Only a terminate still running after 90 s is left behind, with a warning.
  • With a hung terminate the drain takes about 90 s on iOS. On Android the host clear (force-stops, instrumentation wait, port close) can make it longer.
• So the orchestrate may still be running when the next attempt starts. From then on it cannot kill, respawn or start the next attempt's Jet, or cold-boot its device. A Detox launchApp the attempt already had in flight may still finish on the device: the 120 s launch timeout stops the wait, not the launch, and the drain waits for it only within the 30 s orchestrate wait. In detail:
  • An abandoned attempt's launch retry stops first, before an Android cold boot, a kill or a respawn.
  • Its failure handler kills nothing and posts no state to the next attempt's Jet.
  • A launch that finishes late neither posts launch-ok nor signals launch-ready to the next attempt's Jet.
  • The launch-retry and stall kill acts only on a Jet that is still running, and kills its process tree first, with no SIGTERM that would orphan the listener.
  • After a stall kill, the orchestrate failure cleanup does not kill the same Jet a second time (under a launch-retry label) while its exit has not been observed yet. The session records the Jet it killed; a respawned Jet is still killed.
  • The orchestrate wait clears its timer, so a drain no longer logs a timeout after the orchestrate has finished.
  • The Jet session lives in tests/e2e/jetSession.js, with its spawn and kill injected, so these paths are tested with real processes.
• The test-expo and test-rn-bare ios:launch-smoke scripts no longer kill -9 everything on :8081; they stop only the Metro they spawned.
• simulator-logging.sh kills booted log streams only on single-tenant runners (CI=true or RNFB_SIM_UI_RESET=1). Local iOS Jet retries no longer kill the Simulator or DeviceHub UI. android-e2e-detox.sh refuses to run outside CI.

Manual-only recovery

• --include-unrecorded, run in that identity's env, also frees this checkout's unrecorded holders (ports, macOS app by name, and with --devices this checkout's unrecorded emulator on the identity's AVD). Another owner's device and apps are reported and left running.
• --all-slots is the whole-machine wipe.
• yarn test:full --clear-whole-host and --clear-all are removed; both exit 2 and point to the manual command. No script, yarn test:full or CI path passes either flag.

Unchanged

The slot model, the port formula, env names and the yarn tests:e2e:* surface. GitHub-hosted CI never calls check or release; recording is additive there and the fail-fast paths cannot trigger on a fresh runner.

One-time upgrade note

Processes started before this change are unrecorded. The first yarn tests:e2e:check after upgrading may report them owner=this-checkout-unrecorded. Stop them, or run the printed yarn tests:e2e:release --include-unrecorded --only <category> (add --devices for an emulator) once in that identity's env. An unrecorded simulator cannot be traced to a checkout and shows owner=unrecorded; if it is yours, run xcrun simctl shutdown <udid>.

Docs: okf-bundle/testing/running-e2e.md (run record, kill policy, owner labels, manual machine wipe, upgrade note), okf-bundle/testing/e2e-parallel-design.md, okf-bundle/testing/agent-command-policy.md.

Related issues

None. Related PRs in this stack:

• build: size lerna prepare and gradle workers to the CPUs and memory the build can use #9446, build concurrency cap (bottom)
• fix: harden ruby:install, lint:android and host-dependent app tests #9447, tooling and test fixes, stacked on build: size lerna prepare and gradle workers to the CPUs and memory the build can use #9446
• This PR, stacked on fix: harden ruby:install, lint:android and host-dependent app tests #9447
• fix(e2e): harden shared-host e2e scripts for slots, foreign Jet ports and TMPDIR #9448, shared-host e2e hardening, stacked on this PR

Release Summary

None (e2e scripts, tests and contributor docs only).

Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • Yes
• My change supports the following platforms;
  • Android
  • iOS
  • Other (macOS, web)
• My change includes tests;
  • e2e tests added or updated in packages/\*\*/e2e
  • jest tests added or updated in packages/\*\*/__tests__
• I have updated TypeScript types that are affected by my change.
• This is a breaking change;
  • Yes
  • No

Test Plan

Jest

• New e2eRunRecord.test.js covers pid ownership and reuse, device instance tokens (exact AVD match, qemu -port serial), recorded-only release, --include-unrecorded owner and AVD filtering, check owner labels and hints, harness and .jetrc.js fail-fast, owned-only listener recording, failed-start cleanup, the Detox "booted during this run" hook, every harness reboot path re-recording, the cold-boot stop leaving another owner's emulator running, macOS Jet recording itself, simulator-logging gating, the launch-smoke sweep, the CI-only guard and the removed test:full flags exiting 2.

• startPackager.test.js, createE2eDevices.test.js and e2eResourceEnv.test.js cover Metro recorded at spawn, simulator release by owner, and stray-emulator BUSY/INFO.

• Each guard was checked by reverting it and confirming a test fails.

• The same-name macOS app fail-fast test now sets process.title on Linux. Node renames its main thread MainThread there, so pgrep -x never matched the symlinked node. A symlinked sleep is not an option because the runner's uutils coreutils refuses to run under another name. e2eRunRecord.test.js passes 45/45 on macOS and in an ubuntu:26.04 container with Node 24 (the runner image). The CI Jest job is green (119 suites; 1927 passed, 7 skipped).

• New jetAttemptDrain.test.js uses real processes on a real port. It covers the drain stopping the Jet a launch retry respawned (not the stale first one), killing a listener whose parent would exit on SIGTERM and orphan it, treating a signalled but still-running Jet as alive, and abandoning the session. It also checks the harness wiring: the session is kept as the attempt context, the drain abandons before the orchestrate wait, an abandoned session never respawns Jet, and the close-wait timeout names the port's holders. Reverting the harness wiring fails 4 tests; restoring the old !killed liveness check fails 2; putting SIGTERM back before the tree kill fails the orphan test. Passes on macOS and in ubuntu:26.04 with Node 24.

• The same file covers a late attempt-1 orchestrate: a real session is drained, then attempt 2's stand-in Jet holds the same port. Each check is revert-proven: removing it fails one test.

  • A launch retry throws before any kill or spawn.
  • A later orchestrate failure kills nothing.
  • A late launch cannot signal launch-ready.
  • The retry kill leaves the port alone for a Jet that exited by itself.
  • The retry kill kills the tree before an sh root can orphan the listener.
  • The orchestrate wait clears its timer.
  • A launch retry checks for abandonment before the Android cold boot.
  • A late launch posts nothing; a late failure posts nothing.
  • The drain's terminateApp wait ends after its bound, and a later rejection is handled.
  • The drain joins the same terminate after the port close. The join waits until 90 s after the terminate started (not after the join began), and adds nothing when the terminate already finished.
  • The harness uses each of these.

  The orphan-listener tests record the listener's pid and kill it after each test, so a failing run leaves no listener behind (checked by failing them on purpose).

• e2eRunRecord.test.js no longer reads the host's devices. Its fake devices carry a per-run marker. ps (every listing form and per-pid queries), pgrep and pkill hide qemu and launchd_sim processes without it, and adb and simctl list no devices. The pkill stub accepts only -f, a signal and one pattern, and exits 2 on any other flag.

  • Before, a second run of the suite (same fake names, same working directory) or a real emulator on an identity serial failed 2 tests.
  • Also before: the suite's --all-slots wipe test ran the real pkill, which stops every same-AVD emulator on the host, a real one included. Scoped release never pkills by AVD; only the manual --all-slots wipe does.
  • With an unmarked @TestingAVD-1 -port 5558 running, the suite passes and the process survives; without the stubs, the 2 tests fail. New tests check each discovery path, every ps form and the pkill flag handling.

• A stalled Jet whose exit is not yet observed is killed once, not again by the orchestrate failure cleanup; a respawned Jet is still killed. Removing the session's check, or not recording the stall kill, fails the new test.

• The seven affected suites pass on macOS (154 tests) and in ubuntu:26.04 with Node 24.

• Restacked onto build: size lerna prepare and gradle workers to the CPUs and memory the build can use #9446's current tip (fix: harden ruby:install, lint:android and host-dependent app tests #9447 restacked first). fix: harden ruby:install, lint:android and host-dependent app tests #9447 had kept build: size lerna prepare and gradle workers to the CPUs and memory the build can use #9446's first commit as its base, so no commit was dropped. All nine commits of this PR rebased with no conflicts, and git range-diff shows each one unchanged. On the new tip: yarn lint 0, the seven affected suites 0 (154 tests, --detectOpenHandles), and the stack's top (fix(e2e): harden shared-host e2e scripts for slots, foreign Jet ports and TMPDIR #9448) runs full yarn tests:jest 0 and the affected suites 0 in ubuntu:26.04 with Node 24.

• Validation after rebasing onto fix: harden ruby:install, lint:android and host-dependent app tests #9447: yarn lerna:prepare, yarn tsc:compile, yarn tsc:compile:consumer, yarn reference:api, yarn lint, yarn tests:jest (119 suites, 1934 tests), yarn format:js and yarn compare:types all exit 0. The rebase changed no e2e harness or runtime file.

Local serial e2e

Run on one shared host, with no slot environment and the full harness, Android then iOS then macOS. All three passed with 0 failing and coverage merged: Android 1557 passing, iOS 1546, macOS 1340.

• While tests ran, yarn tests:e2e:check --services --platform=<p> from a second shell showed Metro, the emulator suite, Jet, the test app and the device as owner=this-checkout.
• yarn tests:e2e:release --devices --platform=<p> freed exactly the recorded resources and emptied the record. The final release and check were clean.
• Another owner's Android emulator and booted simulators on the same machine were reported INFO and never touched.
• macOS interrupt test: with Jet orphaned, the default yarn tests:e2e:release freed :8090/:8091 and the app with no manual flag.
• Forced iOS Jet retry (serial lane, temporary uncommitted fault injection reproducing the failed CI sequence). A launch retry respawns Jet, then the outer attempt fails as a 1006 disconnect while the orchestrate is still in a launch retry.
  • When the orchestrate hit its next launch retry during the drain, it killed its own Jet and did not respawn.
  • When the orchestrate was still blocked after the drain's 30 s wait, the drain logged stopped the attempt's current Jet, and the port check found :8090 already free.
  • In both cases, :8090 had no listener and no Jet process right after the drain and 15 s later.
  • Afterwards, yarn tests:e2e:release --devices --platform=ios and yarn tests:e2e:check --services --platform=ios were clean. Another owner's booted simulator was reported INFO and left running.

CI e2e workflows

• iOS e2e had not passed on this PR before the drain fix. One earlier run failed in iOS release: two launch timeouts, then Timed out waiting for 127.0.0.1:8090 to close in the outer-retry drain, with a Jet respawned during the drain still holding the port. The other runs were cancelled by newer pushes, or were still queued when a newer push superseded them.
• Before the drain fix, Android e2e passed.
• With the first drain fix, all e2e workflows passed: Android, iOS (debug and release), Other (macOS), Expo iOS link and the RN CLI prebuilt RNCore iOS build. Jest passed too.
  • iOS release: 1543 passing, with no retry.
  • iOS debug: 1544 passing, after one outer retry. Attempt 1 launched, but Jet never reported results, so the harness killed it after the stall timeout. The drain found :8090 already free, and attempt 2 passed.
• With the late-attempt guards, all e2e workflows passed: Android, iOS (debug and release), Other (macOS), Expo iOS link and the RN CLI prebuilt RNCore iOS build. Jest passes too.
  • iOS debug: 1544 passing, with no retry or stall.
  • iOS release: 1543 passing, with no retry.
• With the drain bound and cold-boot guard (before the terminate join), all e2e workflows passed: Android (1556 passing), iOS (debug and release), Other (macOS, 1339), Expo iOS link and both rn-bare iOS builds. Jest passes too.
  • iOS debug: 1544 passing, with no retry or stall.
  • iOS release: 1543 passing, after one outer retry, which exercised the new drain. Attempt 1 hit a launch timeout, respawned Jet for a launch retry, then failed. The drain took 32 s: terminate, then the 30 s orchestrate wait. When attempt 1's retried launch timed out after the drain, it stopped without another launch retry, Jet kill or state post. Attempt 2 passed.
• At the terminate-join tip, all checks pass: Android (1556 passing), iOS debug (1544) and release (1543), Other (macOS, 1339), Expo iOS link, both rn-bare iOS builds, Jest and docs. Several jobs needed a rerun of failed jobs after GitHub infrastructure errors (HTTP 504 on cache and artifact steps, a Gradle download timeout, external doc-link timeouts, one simulator log stream that never became ready). The first iOS debug rerun hit the Jet stall described below in attempt 1; the drain took 2.7 s and attempt 2 then hit a launch timeout. The second rerun passed.
• The iOS debug Jet stall (20 minutes with no tests after launch-ready, recovered by the outer retry) also occurs without this PR: a 2026-11-05 PR run on the same runner image (job 127916619646) shows the same signature. On that image it hit 3 of 5 iOS debug jobs with this PR's changes and 1 of 10 without, which does not separate them. No change in this PR reaches app startup (Metro start, bundle URL, ports, launch args, simulator choice on CI). Other observations:
  • The app, Jet and harness use the same ports (Jet 8090, control 8091, Metro 8081) in stalled and passing attempts, and the retry passes on the same Metro bundle.
  • During the stall, the app was running (its Detox connection stayed open), but it had no connection to Metro or Jet, so its JS never ran.
  • On this runner image, every branch (main included) has Detox pick a second "iPhone 17" simulator rather than the pre-booted one.
  • The outer-retry drain failure this PR fixes also occurs on main: a main iOS release job on the same image failed with the same Timed out waiting for 127.0.0.1:8090 to close.
• At the current tip (restacked onto fix: harden ruby:install, lint:android and host-dependent app tests #9447, plus the stall-kill fix), all 27 checks pass:

Workflow	Job	Result
Testing E2E iOS (42490327114)	iOS (debug, 0)	pass: 1544 passing
	iOS (release, 0)	pass on rerun: 1543 passing, no outer retry
	iOS Release Archive	pass
Testing E2E Android (42490327115)	Android (36, x86_64, google_apis, 0)	pass: 1556 passing
Testing E2E Other (42490327090)	Other (0)	pass: 1339 passing
Testing (42490327131)	Jest	pass: 1974 passed, 7 skipped
Code Quality, Type Comparison, Documentation, Expo iOS Link, both rn-bare iOS builds, Create Test Patches, PR title, codecov		pass

The first iOS release attempt failed in "Detox Test Release": in both outer attempts launchApp timed out after 120000ms although the app's JS had connected to Jet. The launch retry respawned Jet, the still-running first instance connected to it and was dropped (1006) by the retry's terminate, and the 30 s reconnect grace expired (Jet attempt 2 failed (retryable=false, exit=1)). This launch-retry sequence (respawn before terminate, 30 s grace) is the same on main and predates this PR. It involves no stall, so the stall-kill change does not reach it. The rerun passed.

---

Think react-native-firebase is great? Please consider supporting the project with any of the below:

• 👉 Star this repo on GitHub ⭐️
• 👉 Follow React Native Firebase and Invertase on Twitter

[codecov]

Codecov Report

❌ Patch coverage is 85.02994% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.01%. Comparing base (7ee9d26) to head (5fd8751).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #9444       +/-   ##
=============================================
+ Coverage     51.87%   75.01%   +23.15%     
- Complexity        0     2299     +2299     
=============================================
  Files           252      629      +377     
  Lines         11916    41712    +29796     
  Branches       2557     5242     +2685     
=============================================
+ Hits           6180    31286    +25106     
- Misses         5052     8849     +3797     
- Partials        684     1577      +893     

Flag	Coverage Δ
android-native	70.70% <ø> (?)
e2e-ts-android	53.62% <ø> (?)
e2e-ts-ios	52.99% <ø> (?)
e2e-ts-macos	49.31% <ø> (?)
ios-native	79.08% <ø> (?)
ios-ruby	100.00% <ø> (?)
jest	52.40% <85.03%> (+0.54%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mikehardy]

This looks like quite a bit of churn I know but I'm making a concerted effort to solve build/test capacity as a bottleneck

Parallel design had an initial take with slot-based runs and that was successful enough to prompt further investigations. Now I'm scaling parallel design to a federated pool of hosts and with multiple users potentially using the same host now we need a bit more safety with regard to what is started/stopped with regards to our test resources

This PR implements that for RNFB

Quite a few changes stacked up - all infra + test - need to get them through, merging this